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1  Introduction 

Hybrid  control  is  the  control  of  continuous  plants  by  sequential  automata.  This 
usually  means  frequent  changes  in  the  continuous  conventional  control  law  applied 
to  the  plant,  changes  based  on  sensor  measurements  of  the  trajectory.  This  typically 
yields  plant  trajectories  without  smooth  tangents  at  the  discrete  times  when  the 
control  law  ordered  by  the  control  program  changes.  How  and  when  to  make  these 
control  law  changes  is  the  business  of  the  sequential  automaton.  The  question  is  then 
how  should  we  model  this  and  how  can  we  find  control  sequential  automata  to  meet 
a  prescribed  performance  specification. 

We  propose  a  game  framework  for  analyzing,  extracting  and  verifying  digital 
control  programs  for  continuous  plants  by  regarding  such  programs  as  finite  state 
winning  strategies  in  associated  games.  We  call  such  interacting  systems  of  digital 
control  programs  and  continuous  plants  “hybrid  systems”  and  model  them  as  net¬ 
works  of  interacting  concurrent  digital  programs  or  automata,  following  [36],  [37]. 
This  extends  to  hybrid  systems  the  paradigm  introduced  by  A.  Nerode,  A.  Yakh- 
nis,  and  V.  Yakhnis  [38]  for  analyzing  concurrent  digital  programs  meeting  program 
specifications  as  winning  finite  state  strategies  in  associated  two  person  games.  This 
hybrid  game  formulation  is  intended  to  facilitate  the  transfer  of  recent  tools  from 
logic,  concurrency,  and  dynamical  systems  to  extraction  and  verification  of  digital 
control  programs  for  continuous  systems.  Hybrid  Games  also  facilitate  infusion  into 
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hybrid  systems  theory  of  many  ideas  from  the  traditional  differential  game  approach 
to  control. 

The  Basic  Model 

We  now  introduce  our  basic  model  for  Hybrid  Control  which  is  essentially  the 
same  as  the  model  discussed  in  [24].  A  finite  control  automaton  is  an  automaton 
with  finite  input  and  output  alphabets  and  a  finite  number  of  internal  states.  Its 
input  letters  are  fired  by  measurements  of  plant  state.  Its  output  letters  are  control 
signals,  that  is  mode  switches,  for  the  plant  controller.  Our  basic  model  for  a  simple 
hybrid  system  consists  of  the  following. 


1.  A  finite  control  automaton,  which  is  usually  thought  of  as  some  sort  of  logical 
device  or  program  which  malces  inferences  based  on  current  information  about 
the  plant  state  to  deduce  when  to  change  control  laws  for  the  plant.  See  Kohn- 
Nerode  [24],  [25]. 

2.  A  continuous  plant  controller  obeying  the  control  law  currently  supplied  by  the 
finite  control  automaton. 

3.  A  continuous  plant  being  controlled.  We  include  in  the  plant  the  physical  plant 
controller  (actuator),  but  not  the  finite  control  automaton  feeding  control  orders 
(mode  switches)  to  the  physical  plant  controller. 

4.  An  analog-to-digital  or,  equivalently,  a  signal  to  symbol,  converter  supplying  to 
the  finite  control  automaton  as  input  digitized  sensor  data  sampled  from  the 
plant. 

5.  A  digital-to-analog  or,  equivalently,  a  symbol  to  signal  converter  converting  sym¬ 
bolic  control  orders  output  by  the  control  automaton  into  a  control  function  of 
time  regulating  the  parameters  of  the  physical  plant  controller. 


These  elements  are  pictured  in  Figure  1. 
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Figure  1. 

We  summarize  the  essential  features  of  hybrid  systems  model  of  [24].  We  think  of 
the  sequential  control  automaton  as  completing  ‘‘work  cycles”  in  successive  intervals 
Atk  of  time.  During  the  interval  Atk,  a  control  law  Uk  which  is  imposed  by  the  se¬ 
quential  control  automaton  at  the  end  of  the  previous  interval  is  active  in  controlling 
the  system.  Also  the  sequential  control  automaton  is  subject  during  Atk  to  an  input 
Vk  to  the  system.  During  the  first  phase  of  Atk,  the  sequential  control  automaton  is 
accumulating  a  sensor  data  history  s  about  the  system  through  the  analog-to-digital 
converter.  The  sequential  control  automaton  starts  interval  Atk  in  a  certain  initial 
state,  uses  s  to  compute  a  new  control  law  Uk^i  and  a  new  automaton  state  and, 
at  the  end  of  Atk,  it  outputs  Ufc+i  through  the  digital-to-analog  converter  to  the 
plant  controller  for  use  in  the  next  interval  Atk-\^i .  Then  all  processes  start  over  for 
Atk^i,  We  envisage  the  input  as  encoding  all  the  partial  information  available  to 
the  control  automaton  about  the  state  of  the  plant.  A  hybrid  control  run  thus  will 
be  a  possibly  infinite  sequence 
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We  shall  see  that  it  is  very  natural  to  view  such  a  hybrid  control  run  as  play  of  a 
g£une  between  two  players.  Plant  and  Control.  That  is,  Control  and  Plant  alternate 
moves  in  a  game  in  which  Control  moves  by  listing  full  information  about  control  law 


Uk  for  Plant’s  use,  and  then  Plant  moves  by  listing  the  partial  information  Vk  about 
the  plant  state  for  Control’s  use.  The  range  of  values  of  Uk  and  Vk  and  the  relationship 
between  Uk  and  Vk  is  dependent  on  the  particular  application.  Then,  in  the  spirit 
of  [38],  we  can  view  a  successful  sequential  control  automaton  as  implementing  a 
winning  strategy  for  Control.  That  is,  in  any  play  in  which  Control  follows  the 
winning  strategy  and  Plant  plays  according  to  the  rules  of  the  game,  i.e.  follows 
its  differential  equations,  the  plant  trajectories  will  meet  the  desired  performance 
specification. 

Performance  Specifications 

Our  performance  specifications  are  usually  open  sets  of  trajectories.  Quoting  an 
example  of  Kohn,  the  Boeing  737  was  to  be  designed  so  that  if  a  cup  of  coffee  is 
no  more  than  3/4  full  anywhere  in  the  aircraft,  it  never  spills  during  maneuvers. 
This  is  not  a  conventional  optimality  requirement.  It  is  a  “perform  sufficiently  well” 
criterion  which  we  call  an  e-performauice  criterion  or,  alternately,  an  e-optimality 
condition.  For  example,  such  a  criterion  might  require  that  we  produce  a  trajectory 
whose  costs  is  within  a  user  defined  e  of  the  minimum  cost  trajectory. 

Outline  of  Paper 

In  sections  2  and  3,  we  present  game  models  for  extraction,  analysis,  and  veri¬ 
fication  of  control  strategies  for  simple  hybrid  systems.  All  games  will  be  between 
two  players,  Plant  and  Control.  The  objective  of  any  game  is  for  Control  to  force 
Plant  to  obey  its  performance  specification.  In  the  game  model  presented  in  section 
2,  measurements  of  Plant  state  are  made  at  discrete  times  (discrete  sensing)  and 
changes  in  the  control  order  to  the  plant  are  also  made  at  discrete  times  (discrete 
mode  switching).  Such  games  are  an  adaptation  to  hybrid  systems  of  the  games  of 
A.  Nerode,  A.  Yahknis,  and  V.  Yahkinis  [38].  The  latter  were  introduced  to  extract, 
analyze,  and  verify  digital  concurrent  programs. 

In  Section  3,  we  introduce  “continuous  sensing  games”  to  model  plants  charac¬ 
terized  by  continuous  dynamics,  such  as  a  system  of  ordinary  differential  equations, 
with  a  controller  which  continuously  senses  the  plant  state.  We  assume  that  the 
controller  is  allowed  to  reset  the  parameters  of  the  plant  dynamics  at  a  sequence  of 
discrete  times  only.  Such  controllers  allow  us  to  model  directly  analog  sensors  which 
continuously  sense  the  plant  state  and  which  output  exact  real  number  control  pa¬ 
rameters  to  the  physical  plant  controller  at  discrete  times.  In  most  cases,  we  will 
assume  that  the  values  output  by  the  controller  are  purely  digital,  one  of  a  finite 
number  of  control  order,  to  be  implemented  by  a  digital  to  analog  converter.  Even 
in  such  cases,  such  a  controller  must  be  regarded  as  non-digital  if  the  input  values 
allowed  are  exact  real  numbers,  even  if  there  are  only  a  finite  number  of  internal 
states  and  a  finite  number  of  output  control  orders  (mode  switches).  Our  method¬ 
ology  is  to  start  by  extracting  a  continuously  sensing,  discretely  acting,  feedback 
control  function  which  meets  the  performance  specification.  The  discrete  sensing 
games  of  Section  3  can  then  be  thought  of  as  subgames  of  continuous  sensing  games 
in  which  the  information  sensed  between  discrete  sampling  times  is  ignored. 

Thus  continuous  sensing  games  are  a  second  class  of  games  between  two  players, 
Plant  and  Control.  After  each  control  order  is  sent  by  Control  to  the  plant,  Plant 
displays  a  segment  of  a  plant  trajectory  y  which  begins  when  control  order  is  given, 
with  initial  condition  the  plant  state  at  that  time,  and  which  ends  when  the  next 
control  order  is  issued.  We  can  think  of  this  segment  as  a  contiguous  block  of  Plant 


moves,  one  at  each  time  in  that  interval.  In  this  picture  the  Plant  move  at  a  time 
T  >  t  in  the  interval  is  just  the  plant  state  yir).  In  the  same  picture,  a  Control 
move  occurs  at  the  same  instant  r  and  is  either  no  action  or  a  control  order.  We  call 
the  latter  moves  essential  Control  moves.  We  assume  that  essential  Control  moves 
occur  only  at  a  discrete  sequence  of  times.  Each  such  time  is  the  end  of  a  block  of 
Plant  moves.  According  to  this  picture,  in  a  continuous  sensing  game.  Plant  plays 
continuously,  Control  has  continuous  knowledge  of  Plant  moves,  and  Control  makes 
essential  moves  only  once  in  a  while. 

The  motivation  for  introducing  continuous  sensing  games  is  that  they  help  us 
extract  strategies  for  a  digital  controller  which  will  meet  performance  specifications. 
The  idea  is  that  it  often  easier  to  find  a  (non-digitaJ)  continuous  state  strategy 
for  Control  in  a  continuous  sensing  game  which  forces  the  plant  to  meet  perfor¬ 
mance  specifications.  We  then  extract  a  finite  state  strategy  for  a  finite  state  digital 
controller  doing  approximately  the  same  thing  by  approximating  to  the  continuous 
strategy  for  the  non-digital  controller  using  the  Kohn-Nerode  method  of  extracting 
finite  control  automata  from  finite  open  covers. 

In  Section  4,  we  discuss  performance  specifications  and  the  Kohn-Nerode  method 
cover  method  ([24],  appendix  2).  It  works  as  follows.  Suppose  we  are  given  a  con¬ 
troller  which  meet  an  open  specification.  Then  the  Kohn-Nerode  method  takes  an 
open  finite  cover  of  that  controller  within  the  open  specification  and  interprets  it  as: 

1.  A  finite  automaton  with  a  finite  input  alphabet  and  a  finite  state  alphabet. 

2.  A  digital  to  analog  converter. 

3.  An  analog  to  digital  converter. 

4.  A  control  automaton  for  the  plant. 

When  considered  as  a  hybrid  system,  the  plant  plus  the  automaton  derived  from 
the  cover  forces  the  plant  to  obey  the  open  performance  specification.  We  carry 
out  this  process  for  a  simple  model  of  a  water  pump  used  to  maintain  a  certain 
range  of  values  of  the  water  in  a  water  tank.  That  is,  we  shall  explicitly  construct 
a  strategy  for  Control  in  a  continuous  sensing  game  which  models  this  system  and 
then  show  how  we  can  easily  approximate  such  a  strategy  to  construct  a  strategy 
for  Control  in  a  corresponding  discrete  sensing  game.  Finally  we  shall  show  how  this 
enables  us  to  design  a  digital  control  automaton  for  the  hybrid  system  which  meets 
the  performance  specifications  as  well  as  how  to  construct  the  Kohn-Nerode  small 
topologies  for  the  hybrid  system  which  will  verify  the  controllability-observability  of 
the  system  in  the  sense  of  [24]. 

2  Games  with  Discrete  Sensing  and  Discrete  Mode 
Switching 

In  this  section,  we  provide  a  game  setting  for  the  specification,  extraction,  and  veri¬ 
fication  of  digital  control  programs  for  hybrid  systems.  Extracting  a  control  program 
for  a  continuous  plant  which  forces  the  plant  to  obey  a  performance  specification 
is  identified  with  extracting  a  winning  finite  state  strategy  in  an  associated  game. 
The  performance  specification  itself  is  identified  with  a  set  of  acceptable  plant  state 


trajectories.  The  games  introduced  in  this  section  and  the  next  section,  each  have 
two  players,  Control  and  Plant.  In  our  games,  we  represent  the  effect  on  plant  state 
of  unknown  disturbances  and  uncertain  measurements  by  allowing  multiple  legal 
moves  for  Plant.  For  example,  one  source  of  multiple  possible  moves  for  Plant  is 
that,  with  a  given  initial  condition,  each  disturbance  over  a  time  interval  [^,  t-\-  A] 
can  yield  a  different  plant  state  trajectory  over  that  interval  and  hence  a  different 
final  plant  state  at  the  end  of  the  time  interval.  Another  source  of  multiple  Plant 
moves  is  measurement  errors.  We  assume  Control  sends  perfect  information  to  the 
physical  plant  controller,  namely  a  suitable  control  law  for  the  next  interval  of  time. 
However  Plant  sends  imperfect  information  to  the  Control  program,  namely  sensory 
measurements  of  plant  state  with  error.  Thus  our  games  are  games  with  perfect 
information  on  the  control  law  transmitted  by  Control  to  Plant,  but  with  imperfect 
information  on  Plant  state  transmitted  to  Control  by  Plant. 

Our  game  approach  is  different  from  traditional  methods  of  extracting  control 
in  the  presence  of  disturbances  or  measurement  uncertainties.  For  example,  one  tra¬ 
ditional  control  engineering  approach  is  to  start  instead  with  a  deterministic  plant 
model  which  does  not  incorporate  either  disturbances  or  measurement  uncertainties, 
to  proceed  to  extract  a  suitable  control  program  for  the  deterministic  model,  and 
afterwards  to  determine  the  effect  of  small  changes  in  measurements  and  param¬ 
eters  on  observability,  controllability,  and  stability  of  the  hybrid  system.  Another 
approach  is  to  model  the  Plant  by  stochastic  differential  equations  in  the  first  place, 
and  to  look  for  stochastic  control  programs  with  optimal  control  features.  A  third 
approach  is  to  use  a  two  person  differential  game  between  Control  and  Plant  or  be¬ 
tween  Control  and  Disturbance.  This  usually  entails  extracting  continuous  control 
strategies  which  change  control  values  continuously,  based  on  continuous  measure¬ 
ments  of  plant  state.  To  extract  such  a  continuously  sensing  continuously  controlling 
strategy  using  differential  games  usually  requires  elaborate  mathematical  apparatus 
when  it  is  possible  at  all.  Our  games  approach  differs  from  all  three.  Control  strate¬ 
gies  are  not  derived  directly  from  a  deterministic  model.  The  model  does  not  involve 
stochastic  processes.  It  is  a  game  approach,  but  not  the  usual  differential  games 
approach.  In  our  games,  one  player.  Plant,  is  constrained  to  follow  a  differential  or 
difference  equation  guided  by  controls  and  subject  to  disturbances.  The  change  is 
that,  in  our  games,  measurements  of  Plant  state  are  communicated  to  Control  only 
at  discrete  prescribed  times,  while  a  change  in  the  control  function  imposed  by  Con¬ 
trol  on  the  Plant  can  be  imposed  instantaneously.  The  changes  imposed  by  Control 
on  the  plant  are  event-driven  based  on  the  current  state  of  the  control  automaton 
and  the  current  measurement  of  plant  state.  Restricting  Control  in  this  way  is  nat¬ 
ural  if  Control  is  to  be  a  digital  program,  since  a  digital  program  changes  its  state 
based  on  a  discrete  sequence  of  successive  input  symbols  representing  plant  measure¬ 
ments.  Even  if  Control  is  not  restricted  to  a  digital  program  with  finite  alphabets 
and  states,  the  discrete  sensing,  discrete  mode  switching  control  strategies  turn  out 
to  be  useful  as  intermediate  idealized  programs  to  extract  before  refining  them  to 
finite  state  strategies  which  give  controllable-observable  behavior. 

The  system  model  underlying  our  game  is  the  hybrid  systems  model  of  Kohn- 
Nerode,  [24]  and  [25],  to  which  the  reader  is  referred.  The  games  approach  stems  from 
the  Nerode-Yakhnis-Yakhnis  [38]  formulation  of  extracting  concurrent  programs  as 
solving  an  appropriate  game.  The  hybrid  systems  games  were  first  announced  in 


Nerode-YaMinis  [36], [37]. 

Control  automata  which  sense  plant  state  at  discrete  times  but  exercise  control 
over  the  plant  continuously,  with  only  occasional  mode  switching,  operate  in  the  fol¬ 
lowing  way.  Their  input  alphabets,  internal  states,  and  output  alphabets  can  be  any 
finite  or  infinite  nonempty  set.  They  can  be  regarded  as  non-deterministic  automata 
operating  in  continuous  time.  They  change  their  input  alphabet  letter  and  internal 
state  instantaneously  at  a  discrete  sequence  of  time  instants  only,  being  in  the  pre¬ 
vious  automaton  state  in  a  non-empty  open  interval  preceding  each  such  moment. 
These  are  the  moments  when  sense  data  about  the  plant  are  communicated  to  the 
control  automaton.  Only  at  these  times  does  the  control  automaton  instantaneously 
change  its  output  letter,  called  a  control  order.  This  output  letter  is  to  be  interpreted 
in  applications  as  a  control  order  to  the  plant  physical  controller  to  change  the  con¬ 
trol  law  used  in  that  physical  controller.  For  instance,  in  Kohn-Nerode  extraction 
procedure  [26,  15],  this  issued  control  order  is  a  chattering  control  implemented  via 
a  finite  sequence  of  “primitive”  control  actions,  each  specifying  a  physical  controller 
parameter  to  be  used  for  some  period.  Such  a  control  order  is  a  finite  sequence  of 
infinitesimal  generators  of  flows.  Each  flow  is  to  be  followed  in  the  prescribed  order 
for  a  prescribed  relative  duration  of  the  interval  of  time  over  which  this  control  order 
persists. 

In  summary,  control  orders,  or  mode  switches,  are  issued  by  the  control  au¬ 
tomaton  on  an  event  driven  basis  based  on  past  sense  measurements  of  plant  state. 
Although  we  allow  the  set  of  control  automaton  states  to  be  infinite,  in  all  our  ex¬ 
amples  the  automaton  will  be  finite  state,  while  the  input  alphabet  representing 
possible  sense  measurements  will  be  infinite. 

Next  we  describe  the  underlying  assumptions  on  the  plant  model  and  the  control 
automaton  for  our  basic  discrete  sensing  game. 

We  assume  as  a  physical  realizability  requirement  that  the  discrete  times  at 
which  the  the  control  automaton  issues  control  orders,  to  <  <  ^2  <  ^3  <  have 

a  positive  lower  bound  for  the  differences  tt+i  —  ti.  This  usually  called  the  Zeno 
requirement.  We  call  these  sequences  admissible  time  sequences.  In  this  section,  we 
shall  assume  that  for  all  i,  tj+i  —  ti  =  T  is  a  fixed  positive  constant  T.  In  a  later 
section,  this  simplification  is  dropped. 

Plant  model 

Our  basic  assumption  of  the  plant  model  are  the  following. 

1.  We  assume  the  plant  is  modeled  by  an  ordinary  vector  differential  equation 

il(t)  =  /(t,i/(t),u(t),d(t)), 

where  y(t)  is  the  plant  state,  u(t)  is  a  control  function,  and  d(t)  is  disturbance 
function. 

2.  The  time  t  will  range  over  the  real  interval  [0,  oo).  Plant  state  trajectories  p(t)j 
control  functions  of  time  u(t),  and  disturbance  functions  of  time  d(t)  will  be 
defined  on  [0,oo). 

3.  The  function  p  =  y(t),  which  we  call  the  plant  state  trajectory,  takes  values 
in  Xj  the  set  of  plant  states.  There  will  be  a  class  S  of  admissible  plant  state 
trajectories. 


4.  The  function  u  =  u{t)  takes  its  values  in  a  set  U  of  admissible  control  values. 
There  will  a  class  C  of  admissible  control  functions. 

5.  The  function  d  =  d{t)  takes  its  values  in  a  set  D  of  admissible  disturbance  values. 
There  will  be  a  class  V  of  admissible  disturbance  functions  of  time. 

6.  The  sets  of  admissible  plant  states,  control  values,  and  disturbance  values  are 
assumed  to  be  subsets  of  fixed  finite  dimensional  Euclidean  spaces. 

Here  is  the  kind  of  problem  we  want  to  solve.  Suppose  that  a  subset  V  of  the 
plant  states  is  specified,  which  we  call  the  viability  set  Suppose  that  a  subset  of  the 
viability  set  V  is  given,  which  we  call  the  goal  set  G.  We  want  to  extract  a  control 
strategy  which  satisfy  the  following  conditions. 

1.  Starts  the  plant  at  time  to  in  a  prescribed  plant  state  yo  in  the  viability  set  V. 

2.  Ensures  that  at  all  subsequent  times  t,  the  plant  state  y  =  y{t)  is  also  in  the 
viability  set  V. 

3.  Ensures,  as  a  winning  condition  for  the  game,  that  the  plant  state  enters  the 
goal  set  G  by  a  designated  time.  (Alternative  winning  condition  might  that  the 
plant  state  eventually  enters  the  goal  set  G  or  the  plant  state  must  enter  G  in  a 
certain  time  interval  (t/i ,  t/a ) .) 

All  the  control  automaton  can  do  at  time  t  is  to  define  the  control  law  for  the 
next  interval  to  be  incorporated  into  the  control  function  of  time.  But  the  control 
automaton  has  no  infiuence  over  the  disturbance  function  of  time  d  =  d{t)  encoun¬ 
tered.  Thus  the  control  automaton  must  select  the  next  control  law  in  such  a  way 
as  to  keep  the  plant  state  in  the  viability  set  V  and  lead  to  the  goal,  at  a  designated 
time  or  eventually,  as  required,  no  matter  what  admissible  disturbance  function  is 
encountered. 

All  the  information  the  control  automaton  has  available  to  decide  what  new 
control  to  impose  is  its  own  automaton  state  plus  the  current  sensor  measurements 
of  plant  state. 

In  summary,  the  problem  is  to  construct  a  control  automaton  which,  given  both 
its  current  state  and  measurement  of  plant  state  at  the  end  of  the  current  interval, 
changes  to  a  new  state  and  outputs  new  control  law  to  be  used  for  the  next  interval 
such  that  if  the  plant  state  starts  at  time  to  in  the  viability  set  V,  with  a  prescribed 
initial  control,  the  plant  state  trajectories  stay  entirely  within  V  and  either  enters 
the  goal  set  G  by  a  prescribed  time  or  alternately  eventually  enters  the  goal  set  G. 
Admissible  Control  Functions 

Assume  that  the  set  of  admissible  control  functions  C  is  a  set  of  functions  which 
contains  a  set  of  functions  Go  from  [0, 1]  to  f7.  If  a  <  6,  and  c  is  a  control  law  from  Go, 
then  the  corresponding  control  law  on  [a,  6]  is  defined  as  the  function  c{{t—a)/{b—a)). 
Our  minimal  assumption  on  the  set  of  admissible  control  laws  C  is  the  following. 

Suppose  that  u  maps  [0,oo)  into  U  and  there  exists  a  sequence  of  times 
to  <ti  <t2  <h  <  ....  such  that  for  every  n,  there  is  a  function  c  in  Go  for 
which  c  corresponds  to  u  restricted  to  [tn,tn+i).  Then  u  is  in  C. 

We  also  assume  a  similar  relation  between  the  set  Do  of  admissible  disturbances 
mapping  [0, 1]  to  F,  and  the  set  of  admissible  disturbance  functions  V  of  time  map¬ 
ping  [0,  oo)  to  D.  We  do  not  specify  exactly  the  closure  conditions  on  C  or  on  P.  In 


some  contexts,  C  is  the  set  of  all  continuous  functions,  V  is  the  set  of  all  measurable 
functions,  etc. 

Uniqueness  of  Plant  State  Trajectories 

We  assume  that  each  admissible  control  function  and  disturbance  function  gives 
rise  to  a  unique  plant  state  trajectory.  That  is,  suppose  the  classes  C,  V  and  the  plant 
function  /  are  given.  We  shall  assume  that  our  plant  model  satisfies  the  following 
condition. 

Given  an  admissible  control  function  u,  an  admissible  disturbance  function 
d,  an  admissible  plant  state  yo>  a  time  toj  there  exists  a  unique  admissible 
plant  trajectory  function  y  =  y{t)  with  domain  [to,oo)  such  that  y(to)  =  2/o 
and  for  al\t>  to,  y  satisfies 

y{t)  = 

Bounded  Measurement  Error 

We  assume  that  if  y  is  a  plant  state  and  m  is  its  measurement,  then  there  exists 
an  e  >  0  such  that  ly  -  m|  <  e. 

We  are  now  in  position  to  define  the  legal  positions  of  the  a  discrete  sensing 
game.  Assume  that  we  are  given  a  fixed  admissible  time  sequence  U  =  to-\-  lAt, 
Game  Positions 

Each  (legal)  position  in  the  game  will  be  a  sequences  of  moves 

moj  Cq,  TTlx ,  Cl , ...,  TTlfi,  Cfi 

alternating  between  the  players,  with  Plant  moving  first.  Plant  makes  even  numbered 
moves.  Control  makes  odd  numbered  moves.  Here  is  the  simultaneous  inductive  def¬ 
inition  of  the  notion  of  (legal)  positions  of  the  game,  and  of  the  trajectory  associated 
with  a  position. 

1.  Suppose  that  p  is  the  opening  (null)  position.  Plant  may  choose  as  a  move  any 
admissible  Plant  state  mo-  We  call  any  admissible  state  x  such  that  ja:  -mo|  <  e 
a  trajectory  associated  with  that  position.  That  is,  we  interpret  each  such  a;  as  a 
possible  measurement  of  true  Plant  state  mo  at  time  to^  and  also  as  a  degenerate 
trajectory  starting  and  ending  at  to- 

2.  Suppose  that  p  is  a  position  p  of  odd  length.  Control  may  choose  as  move  any 
admissible  control  law  c  fi:om  C.  The  trajectories  associated  with  position  pc  are 
the  same  as  the  trajectories  associated  with  p. 

3.  Suppose  that  p  =  p'c  is  a  non-null  position  of  even  length  with  c  its  last  move 

made  at  time  t*.  Inductively,  suppose  we  have  already  defined  the  set  of  all 
plant  trajectories  associated  with  p'.  Then  Plant  may  choose  as  move  at  position 
p  any  m  such,  there  exists  a  trajectory  associated  with  p  whose  end  state  z 
has  \z  —  7n\  <  e.  Inductively,  we  define  the  trajectories  associated  with  the 
position  p,  m  as  those  trajectories  extending  at  least  one  trajectory  associated 
with  p  to  a  trajectory  defined  also  on  which  solves  on  that  interval  the 

same  differential  equation,  using  the  control  function  of  time  on  that  interval 
associated  with  c  and  using  some  disturbance  function  of  time  on  that  interval 
associated  with  an  admissible  disturbance.  Thus  for  any  n,  if  Control  makes 


move  c  at  time  t„,  then  the  control  function  of  time  applied  to  the  Plant  in 
[*n.<n+i]  is  c(<  -  tn)/itn+i  “  *n))-  If  d  is  in  Do,  the  corresponding  disturbance 
function  of  time  on  the  time  interval  is  d{{t  —  tn)/{tn+i  — 1„)).  Due  to 

our  trajectory  field  assumption,  on  [i„,tn+i])  there  is  a  unique  plant  trajectory 
y  =  yx,c,d,t„  determined  by  the  plant  state  x(t„)  on  the  trajectory  x{t)  associated 
with  p,  together  with  the  control  law  c  firom  Co  and  the  disturbance  d  from  Do 
and  the  differential  equation. 

2/(*)  =  /(*,!/(«).«(*).  d(f)). 

For  any  x  (in  a  Euclidean  space)  define  Bo/Z(x,e)  =  {y  £  :  \x  —  y\  <  e}.  For 

any  subset  Y  the  space,  define  Ball{Y,e)  =  UxeYBall{x,e)  The  plant  moves  m 
can  then  be  described  as 

{z  6  Ball{PlantStates,e)  |  (3  v  €  D){\yx^c,Vyti{U-^i)  ”  ^1  <  e}. 

We  define  the  set  of  finite  plays  of  the  game  to  be  the  set  of  legal  positions 
described  above.  An  infinite  play  is  an  infinite  sequence,  each  finite  initial  segment 
of  which  is  a  finite  play.  Trajectories  associated  with  infinite  plays  are  similarly 
defined. 

There  are  alternate  definitions  of  “winning  the  game” ,  depending  on  what  control 
problem  has  to  be  modeled.  For  example,  given  the  basic  control  problem  of  trying 
to  bring  the  plant  firom  some  initial  point  x  in  the  viability  set  y  to  a  point  in  the 
the  goal  set,  G,  the  appropriate  notion  of  “winning  the  game”  is  as  follows. 

Winning  a  Play 

We  say  that  Control  wins  play  /x,  or  alternately  that  /x  is  a  winning  play  for 
Control,  if 

1.  /X  is  a  finite  play. 

2.  For  the  last  Plant  move  m  of  /x,  BaZZ(m,  e)  is  a  subset  of  the  goal  set  G. 

3.  All  states  traversed  along  all  plant  trajectories  associated  with  /x  are  in  the 
viability  set  V, 

We  note  that  there  are  other  natural  notions  of  winning  plays  depending  on  the 
control  problem  to  be  solved.  For  example,  we  might  define  Control  as  winning  a 
play  if  all  associated  plant  trajectories  stay  in  an  e  neighborhood  of  a  fixed  curve 
in  plant  state  space.  For  example  if  is  optimal  plant  trajectory  with  respect  to 
some  Lagrangian  L,  then  we  might  take  the  viability  set  V  for  this  example  as  the 
set  of  pairs  (a:,  t)  such  that  a;  is  a  plant  state  and  t  is  a  time  and  \x  -  <  e.  Our 

games  can  easily  be  modified  to  deal  with  a  variety  of  control  problems. 

A  strategy  for  Control  is  a  map  F  from  the  set  of  positions  of  the  game  of  odd 
length  into  Go.  The  idea  here  is  that,  given  a  play  mo,  cq, mi, ci, m„,  the  function 
F{mo,co^rni,ci,,..,mn)  =  Cn  determines  the  next  move  of  Control.  We  say  that  a 
play  P  =  mo,co,mi,ci,...,mn,Cn  is  generated  by  the  strategy  F,  or  that  p  is  play 
in  which  Control  follows  F,  if  for  all  i,  Ci  =  F(mo,co,mi,ci,...,mi).  Strategies  for 
Plant  can  be  defined  in  a  similar  manner. 

The  notion  of  which  strategies  are  winning  for  Control  depends  on  the  definition 
of  what  it  means  for  Control  to  win  the  game.  In  the  remainder  of  this  section  a 
strategy  F  for  Control  is  a  winning  strategy  if,  whenever  Control  follows  F,  Control 


will  eventually  reach  a  winning  position,  no  matter  what  initial  position  mo  in  the 
viability  set  V  is  chosen  by  the  Plant  to  start  the  game  and  no  matter  what  the 
subsequent  moves  of  Plant  are. 

An  automaton  strategy  for  Control  is  an  automaton  with  the  following  properties. 

1.  The  set  of  automaton  states  S  is  any  non-empty  set. 

2.  The  automaton  input  alphabet  is  Ball{V,e)  where  V  is  the  viability  set. 

3.  The  automaton  output  alphabet  is  Co. 

4.  The  automaton  transition  table  M(s,m)  and  its  output  function  ff(s,m)  are 
such  that  the  output  is  produced  simultaneously  with  the  automaton  shifting  to 
its  new  state  r  € 

We  call  such  an  automaton  a  control  automaton. 

We  say  that  a  Control  automaton  strategy  generates  a  play 

-  ^0)  )  Cl  j  •••)  Cyj. 

if 

1.  CO  =  H{3o,Tno)  and  the  next  control  automaton  state  is  Si  =  M{so,mo)  where 
So  is  the  initial  state  of  the  automaton. 

2.  At  time  tjfe  =  to  +  kT  in  a  position  with  last  Control  move  Ck  and  last  Plant 
move  mjb,  the  next  control  automaton  state  is  Sfe+i  =  M{sk,mk)  and  the  next 
control  law  is  =  H{sk^rnk)- 

We  say  that  an  automaton  strategy  for  Control,  or  equivalently  control  automa¬ 
ton,  is  winning  for  Control  if  whenever  Control  generates  plays  following  the  control 
automaton  as  described  above,  then  Control  will  reach  a  winning  position,  no  matter 
what  initial  move  mo  in  the  viability  set  V  is  chosen  by  the  Plant  to  start  the  game, 
and  no  matter  what  the  subsequent  moves  of  Plant  are. 

Finite  Input-Output  Alphabet  Games 

Real  digital  controllers  are  finite  state  machines  with  finite  input  and  output 
alphabets.  We  adapt  our  definitions  for  using  such  controllers  as  Control  strategies. 
First  let  F'  be  a  finite  subset  of  Ball{V,e)  such  that 

(Vj/ev^KVen  (|y-y'l<<5)- 

Then  if  we  we  replace  V  by  F'  in  all  the  definitions  above  and  we  assume  that  the 
set  of  controls  Co  is  finite,  then  we  have  defined  a  subclass  of  games  which  we  call 
finite  alphabet  discrete  sampling  games.  For  these  games  the  control  automata  are 
always  finite  automata. 

We  end  this  section  by  giving  an  explicit  example  of  how  a  problem  that  has  been 
studied  in  the  literature  can  be  expressed  in  game  language. 

Railroad  Problem:  This  is  a  variation  of  a  problem  considered  by  Schneider 
and  Marzullo  [32],  Here  the  plant  is  a  train  whose  plant  state  space  consists  of  pairs 
(j/,  C)  where  y  is  a  position  on  a  line  and  C  is  the  train  velocity  at  that  position.  Thus 


the  plant  space  is  a  subset  of  a  2-  dimensional  Euclidean  space.  The  plant  dynamics 
are  given  by 

fy  =  c 

=  u-hu 

where  u  is  a  control  parameter  and  v  is  the  train  engine  acceleration.  Sensors  can 
measure  the  train  position  and  velocity  with  known  error  bounds.  We  assume  that 
there  is  a  common  bound  e  on  uncertainty  in  position  and  velocity.  There  is  a 
viability  set  V  based  on  a  partition  of  the  track  into  contiguous  blocks.  For  each 
block,  there  are  regulations  requiring  that  certain  minimum  and  maximum  velocity 
bounds  be  respected  when  the  train  is  on  that  block.  That  is,  suppose  there  are 
n  >  1  blocks,  and  each  block  is  defined  by  its  beginning  position  and  its  length 
(6<,/ni),  0  <  t  <  n  ~  1.  The  corresponding  velocity  bounds  are  (mirii^maxi).  Thus 

V  =  {(2/j  C)  \bi  <y  <bi  +  Irii  C  e  [mini,  maxi],  0  <  z  <  n  -  1}. 

The  velocity  is  assumed  to  be  in  a  fixed  direction  along  a  straight  railroad  line. 
Hence  all  positions  of  the  train  are  in  that  direction  firom  the  initial  position  0. 

The  goal  set  is  defined  by  a  distance  D  >  0  from  the  origin  where 

D<  Im. 

0<i<n-l 


That  is. 


G  =  Ball{{D,e)  x  {0}. 

The  problem  is  to  guide  the  train  to  stop  within  the  interval  [D  —  e,D  e]  while 
satisfying  the  blocks  constraints  along  the  way. 


3  Continuous  Sensing,  Discrete  Mode  Switching 

In  this  section,  we  define  a  second  class  of  games  which  we  call  continuous  sensing 
games.  Throughout  this  section,  we  keep  the  same  set  of  assumptions  on  the  plant 
model  and  continue  that  same  notation  as  used  in  section  2.  Our  basic  underlying 
model  is  a  hybrid  system  in  which  the  plant  state  is  sensed  continuously,  but  new 
control  orders  (mode  switches)  are  issued  at  discrete  times.  We  illustrate  this  idea 
with  the  following  simple  example. 

Water  Level  Monitor 

Our  water  level  monitor  is  a  generalization  of  an  example  analyzed  in  [1].  The 
plant  consists  of  a  water  pump  and  a  water  tank.  The  controller  issues  control  orders 
to  turn  on  or  turn  off  to  the  pump.  The  plant  state  is  the  pair  consisting  of  the  water 
level  y  >0  and  the  state  of  the  pump  pmp  €  {on,o//},  telling  whether  the  pump 
is  on  or  off.  The  state  of  the  pump  determines  the  dynamics  of  the  water  level.  We 
assume  that  the  water  level  y  satisfies 

.  _  r  fi{y)  if  the  pump  is  on 
^  “  I  f2{y)  if  the  pump  is  off 

where  fi  and  /2  are  continuous  functions  such  that 


(1) 


0  <  a'  <  fi{y)  <  a,  for  all  y  and 
0  >  -&'  >  /2(y)  >  -6,  for  all  y. 


Moreover,  we  shall  assume  that  there  are  constants  Li  and  L2  such  that  for  all 
X  and  y,  \fi{x)  -  fi{y)\  <  Li\x  -  y\  for  i  =  1, 2. 

Thus  the  states  of  the  plant  can  naturally  be  partitioned  into  two  disjoint  classes; 
one  class  where  the  pump  is  on  and  the  other  class  where  the  pump  is  off.  The 
controller  has  two  control  actions  {pon,poff}  which  cause  transitions  between  the 
two  classes  of  plant  states.  We  assume  that  the  transitions  take  time  up  to  d  >  0, 
the  delay,  to  complete.  That  is,  until  a  transition  has  been  completed,  the  pump 
is  regarded  as  being  in  its  preceding  state  and  the  corresponding  equation  for  the 
water  level  d5mamics  applies. 

Our  controller  has  only  two  states:  {son,  so//}.  The  action  of  the  controller  is 
the  following.  If  the  controller  receives  a  measurement  y  of  the  current  water  level 
when  the  controller  is  in  state  son,  then  it  checks  whether  the  condition  y  >  g  holds 
where  p  >  0  is  a  given  constant.  If  the  condition  holds,  then  the  controller  outputs 
a  order  poff  to  cause  the  pump  to  be  turned  off  and  the  controller  shifts  to  the 
state  so//  instantaneously.  Otherwise,  the  controller  remains  in  its  state  son  and 
outputs  no  order  to  the  pump.  If  the  controller  is  in  the  state  so//  and  receives 
a  measurement  y,  then  it  checks  whether  the  condition  y  <  h  holds  where  h  >  0 
is  another  constant.  If  the  condition  y  <  h  holds,  then  the  controller  outputs  the 
order  pon  to  cause  the  pump  to  be  turned  on  and  instantaneously  shifts  to  the  state 
son.  Otherwise,  the  controller  remains  in  the  state  so//  and  outputs  no  order  to 
the  pump. 

We  note  that  while  the  controller  instantaneously  shifts  to  a  new  state,  the  pump 
does  not  instantaneously  change  its  corresponding  state,  so  the  controller  may  lose 
the  natural  correspondence  between  its  state  and  the  state  of  the  pump.  Note  also 
that  the  controller  is  not  digital,  since  it  is  expected  to  act  at  the  exact  instant  when 
the  water  level  satisfies  the  conditions  causing  the  controller  to  shift  states  and  the 
water  level  is  measured  continuously. 

The  controller  and  the  plant  interact  forever.  We  wish  to  find  those  values  of 
{g,  h)  which  will  guarantee  that  the  water  level  is  maintained  forever  between  two 
constants  0  <u  <v.  That  is,  we  want  to  design  our  controller  to  pick  (y,  h),  so  that 
at  all  times  t,u<  y{t)  <  v. 

Formally,  a  plant  state  is  a  pair  (y(t),  z{t)),  where  y{t)  is  the  water  level,  z{t)  =  1 
if  the  pump  is  on  at  time  t,  and  z{t)  =  0  if  the  pump  is  off  at  time  t.  The  control 
parameter  takes  on  only  two  values,  0  and  1,  where  0  indicates  that  the  pump  has 
been  told  to  turn  off  and  1  indicates  that  the  pump  has  been  told  to  turn  off. 
There  is  no  disturbance.  The  space  of  control  laws  is  the  set  of  piecewise  constant 
functions  with  values  in  17  =  {0, 1}.  The  dynamics  of  the  plant  has  a  form  given 
by  (1).  The  conditions  on  the  /»  which  ensure  that  the  systems  always  has  unique 
fully  extendible  trajectories  for  any  given  initial  condition,  given  at  the  end  of  the 
section,  are  satisfied. 

This  ends  temporarily  our  discussion  of  the  the  water  tank  example.  We  go  on 
to  the  definition  of  a  general  class  of  games  which  will  describe  examples  like  this 
one. 

Next  we  present  two  equivalent  game  models  for  continuous  sensing  games. 


Game  Model  I 

We  begin  with  a  plant  given  by  an  ordinary  differential  equation  with  control 
and  disturbance.  We  consider  the  set  of  plant  trajectories  that  begin  at  a  time 
at  a  points  xo  and  satisfy  the  plant  dynamics  described  at  the  previous  section  for 
some  admissible  set  of  control  functions  C  and  some  admissible  set  of  disturbance 
functions  P.  We  write  TRAJ  for  the  set  of  all  functions  Y{t)  :  [io,oo)  X  such 
that  there  exists  a  control  function  u  e  C  and  a  disturbance  function  v  e  T>  such 
that 

1.  Y (to)  =  xo  £  X  and 

2.  F(i)  =  f{t^Y{t),u{t),d{t))  for  sl\t>  to. 

We  are  assuming  that  there  is  a  unique  Y  G  TRAJ  corresponding  to  any  choice  of 
*0,  wW,  and  d{t). 

Here  is  the  game.  There  are  two  players:  Plant  and  Control.  Plant  moves  are 
taken  from  X  but  their  choice  is  governed  by  certain  members  of  TRAJ.  Control 
moves  are  taken  from  the  set  (C  U  {no  action})  x  [0,  oo).  Suppose  the  game  starts  at 
time  t*.  The  exchamge  of  moves  between  Plant  and  Control  results  in  a  function  of 
time 

Kt)  =  {y{t),4t)) 

where  y{t)  G  X  and  z{t)  G  (C  U  {no  action})  x  {t}.  A  value  of  such  function  at 
time  Hs  a  pair  of  the  last  plant  state  y{t)  observed  and  the  corresponding  Control 
move  which  we  regard  here  as  occurring  instantaneously.  To  determine  its  next  move 
z{t)j  Control  may  utilize  all  values  of  y(r)  at  all  times  r  up  to  and  including  t.  We 
call  such  a  function  ^  a  play,  if  the  following  is  true.  There  is  a  strictly  increasing 
sequence  of  times  {tk  :  k  >  0,to  =  t*}  such  that  for  every  A:  >  0,  z{tk)  £  C  x 
and  for  every  t  G  z{t)  =  {no  action, t)  and  y{t)  =  Ifc(t),  where  i^C) 

is  a  member  of  TRAJ  determined  by  tkyXo  =  y{tk),  the  control  law  in  z{tk),  and 
some  admissible  disturbance  d{t)  G  D.  That  is,  if  z{tk)  =  {uk{-),tk),  then  Yk{t) 
satisfies  that  Yk{tk)  =  y{tk)  and  Yk{t)  =  f{tyYk{t),Uk{t),  {t))  for  all  tk  <t  for  some 
admissible  disturbance  function  d{t).  We  call  the  moves  of  Control  at  the  times  tk  for 
k>0  essential  Control  moves  and  the  moves  at  the  time  t  ^  {tk  :  k  >0},  i.e.  where 
z{t)  =  (no  action,  t),  inessential  Control  moves.  If  the  sequence  {tk  :  k>  0,  to  =  t*} 
is  finite  with  the  last  index  being  n,  we  put  tn+i  =  oo  and  the  above  definition  of  a 
play  applies. 

Definition  1.  Call  a  sequence  {tk  i  k>  0}  realizable  if 

inf  {{to  ~  t*),  {tk+i  -tk):k>0}>0. 

Call  a  play  of  the  game,  realizable  if  the  sequence  of  instances  of  essential  Control 
moves  in  it  is  realizable.  We  will  consider  that  the  plays  which  are  not  realizable  are 
lost  by  Control. 

Game  model  II 

Next  we  will  describe  the  plays  in  our  continuous  sensing  game  in  a  slightly 
different  but  equivalent  way  in  order  to  bring  out  the  resemblance  with  those  games 
in  which  moves  alternate  in  discrete  time.  Only  the  essential  control  moves  will  be 


displayed  in  plays.  Assume  that,  at  the  start  of  the  game  the  time  is  t*,  the  plant  state 
is  X*  e  X,  and  the  initial  control  law  is  We  define  a  block  to  be  a  contiguous 
segment  of  a  play  over  a  right-open  interval  of  time  where  the  corresponding  Control 
moves  are  inessential  except  for  the  leftmost  Control  move.  A  block  may  be  infinite 
if  there  is  no  essential  Control  move  after  it.  In  presenting  a  block,  we  suppress 
the  inessential  Control  moves  in  it  and  we  specify  the  moves  of  Plant  by  giving 
the  element  Y  €  TRAJ  that  determines  its  moves  in  the  segment.  We  remove  the 
Control  move  from  the  leftmost  pair  of  moves  in  the  block  and  place  it  in  firont  of  the 
block  not  regarding  it  as  a  part  of  the  block.  A  play  of  the  game  is  thus  represented 
as  a  sequence  of  blocks  alternating  with  a  sequence  of  essential  Control  moves.  Finite 
sequences  of  this  sort  are  called  the  positions  of  the  game.  A  play  is  a  sequence  of 
positions  such  that  each  next  position  extends  the  preceding  one.  We  will  describe 
all  admissible  positions  by  means  of  induction  on  the  length  of  positions.  We  will 
simultaneously  define  by  induction  a  segment  of  the  plant  trajectory  corresponding 
to  a  position  in  the  game.  Thus  we  will  define,  by  induction  on  n,  the  positions 
the  plant  trajectory  segment  7n  corresponding  to  the  position  Pn,  and  the  right  ends 
t{n)  of  the  domains  of  7n. 

(1)  n  =  1. 

Then  we  let  pi  =  The  corresponding  segment  71  of  the  plant  trajectory  is 

a  single  point  {t*,x*),  i.e.  71  (t*)  =  x*.  We  denote  the  right  end  of  the  time  interval 
of  71  as  t(l)  =  t*. 

Next  suppose  that  the  positions  p„-i  corresponding  to  n  —  1  successive  admissi¬ 
ble  moves  are  defined  along  with  the  corresponding  plant  trajectory  7„_i  which  is 
defined  over  the  interval  [t*,t(n  -  1)]. 

(2)  n  =  2  •  A:  4-  2. 

Suppose  pn-i  =  {u*{.),t*)  ■  Bo- zo- ...  •  Bk-  1  •  Zk-i-  Then  the  next  admissible 
block  Bk  of  Plant  moves  is  specified  by  any  member  i*  :  [f(n — 1),  00)  X  of  T  RAJ 
such  that  Yk(t{n  -  1))  =  7„_i(f(n  -  1))  and  satisfies 

Ykit)  =  f{t,Ykit),Uk-i{t),dit))  for  all  t>t{n-  1) 

where  Uk-i{-)  is  the  control  law  Uk-i{-)  that  occurs  in  the  last  Control  move  Zk-i 
and  d(.)  6  V.  The  plant  trajectory  corresponding  to  the  pn  is  the  function  7„  : 
[t*,  00)  -4  X  defined  by 


7„_i(t)ift  6 
Ykit)  if  t>t{n-  1). 


1)] 


(3)  n  =  2  •  fc  -I-  3. 

Suppose  pn-i  =  ■  Bo -Zo-...-  Bk-i  -  Zk-i  -  Bk.  Then  a  position  of  length 

n  extending  p„_i  is  of  the  form,  p„  =  p„_i  •  Zk  where  Zk  is  any  Control  move  of  form 
such  that  tk  >  tk-i  and  Zk-i  =  We  then  put  t{n)  =  tk  and 

7„  equal  to  7„_i  restricted  to  the  interval  [t*,t(n)]. 

An  infinite  sequence  of  positions,  linearly  ordered  by  extension,  defines  an  infinite 
play.  All  finite  sequences  of  the  form 


t*)  -  Bo-  zq-  ...  -  Bk-i  -  Zk-i  -  Bk 


axe  plays  too.  Plays  which  are  not  realizable  in  the  sense  of  definition  5.1  are  regarded 
as  lost  by  Control. 

It  is  easy  to  see  that  there  is  a  natural  bijection  between  the  plays  of  Game  Model 
I  and  the  plays  of  Game  Model  II. 

Remark  For  convenience  of  notation,  we  will  suppress  the  symbols  for  blocks 
Bk  of  Plant  moves  and  use  instead  the  plant  trajectory  Yk  which  specifies  the  block. 
We  also  suppress  the  first  move  in  the  plays  described  inductively  above  because  we 
regard  it  as  fixing  the  game.  That  is,  the  initial  move  simply  corresponds  to  giving 
initial  settings  of  the  plant,  including  initial  control  parameter  values.  Thus  we  will 
denote  a  play  by  a  sequence  of  the  form 

^0,  Yi,  ^1, ...,  Yfc,  .... 

According  to  our  definition,  each  of  the  trajectories  Yi  :  [ti_i,oo)  ->  X  is  infinite. 
(Here  we  make  the  convention  that  =  t*).  Of  course,  in  the  case  when  there 
is  another  essential  control  move  after  time  we  only  use  the  finite  trajectory 
segment, 2/i  =  F*  restricted  to  to  determine  the  final  plant  trajectory.  Thus 

an  even  more  compact  notation  for  a  play  is  a  sequence  of  the  form 

2/0)  *2^0)  2/l)  ■^1 )  •••»  Vky  •  •  •  • 

We  note,  however,  that  this  last  notation  could  be  misleading  since  it  makes  it  appear 
that  the  time  of  the  next  essential  move  of  Control  is  part  of  the  previous  move  of 
Plant.  A  move  of  Plant  does  not  force  the  timing  of  the  next  essential  Control,  this 
is  forced  by  Control’s  strategy. 

A  winning  condition  for  Control  is  a  set  of  realizable  plays  whose  corresponding 
plant  trajectories  7  satisfy  the  performance  specification  imposed  on  the  hybrid 
system.  For  example,  in  the  water  level  game,  the  performance  specification  is  that 
for  alH,  u  <  y{t)  <  v. 

We  are  interested  in  existence  of  winning  strategies  for  Control  in  such  a  game. 
Intuitively,  a  strategy  is  any  kind  of  systematic  behavior  of  Control  in  a  game  which 
determines  its  next  move  on  the  basis  of  the  knowledge  of  the  previous  moves  of  the 
players  in  a  play.  A  winning  strategy  is  a  behavior  that  is 

1.  defined  for  all  positions  which  are  reached  while  using  such  a  behavior  and 

2.  all  plays  generated  by  such  a  behavior  are  winning  for  Control. 

Following  Buchi,  we  consider  a  description  of  such  a  behavior  by  means  of  an  au¬ 
tomaton  whose  input  alphabet  is  the  set  of  the  opponent’s  moves  X,  and  whose 
output  alphabet  is  (C  U  {no  action})  x  [0,oo).  We  do  not  require  at  this  point  that 
either  of  the  alphabets  be  finite  and  we  do  not  require  that  the  automaton  set  of 
states  be  finite. 

Such  an  automaton  is  to  be  capable  of  continuously  reading  its  input.  At  the 
end  of  this  section,  we  give  a  formal  definition  of  a  continuous  input-discrete  output 
automaton  and  describe  sufiicient  conditions  for  such  an  automaton  to  generate 
exclusively  realizable  plays. 

Modeling  Delays  We  can  model  a  delay  in  resetting  the  next  control  law  to 
be  imposed  on  the  plant.  Such  a  delay  may  depend  on  the  current  control  and  on 
the  next  control  law.  We  assume  that  the  two  laws  determine  an  upper  bound  d 


for  the  reset  time  interval-  We  model  this  in  the  game  rules  for  Plant.  If  Control 
makes  a  move  {uk{*)^tk)i  we  view  this  as  an  order  to  reset  the  current  control  law  to 
Ufc(.).  The  actually  time  r*.  at  which  we  change  to  the  new  control  law  will  be  some 
'Tjb  G  [ijfc,  ifc  -h  d]-  The  Plant  moves  which  form  the  next  block  will  be  of  the  form 


Zk{t) 


for  tk  <t  <Tk 
for  t>Tk 


where  Zk  mapping  [rjk,oo)  into  the  plant  states  is  the  unique  trajectory  determined 
by  the  initial  condition  Zk{Tk)  =  Yk^i{rk),  the  control  law  and  an  admissible 
disturbance  function  d(.).  All  the  rest  is  as  in  the  preceding  definition  of  the  game, 
except  that  it  is  the  reset  times,  rather  than  the  time  Control  moves,  which  determine 
the  plant  trajectory  corresponding  to  a  play.  The  realizability  of  a  play  is  determined 
by  the  sequence  of  reset  times,  so  we  must  assume  that  for  any  fc,  f  jb+i  —  tk  exceeds 
the  positive  lower  bound  d. 


3.1  Uniqueness  and  Extendibility  of  Plant  Trajectories 

Next  we  discuss  sufficient  conditions  for  the  plant  trajectory  corresponding  to  a 
play  to  be  unique  and  continuous.  For  an  example  of  such  a  condition,  consider  the 
Caratheodory  conditions  ([12])  to  be  imposed  on  the  plant  model  /  modified  to  allow 
control  and  disturbance  parameters. 

Caratheodory  Conditions 

We  consider  plants  modeled  by  the  vector  ordinary  differential  equation 

y  =  fit, y,u,d) 

where  t  G  [to>  oo)?  x  £  X,  u  d  G  U,  and  which  satisfy  the  following  conditions. 

CC  1:  For  every  u  and  for  almost  all  t,  the  function  /(t,  or,  u,  d)  is  continuous 
in  {x,d). 

CC  2:  For  every  u  and  every  a;,  the  function  /  is  measurable  in  (t,  d). 

CC  3:  For  every  n,  there  is  function  m(.)  over  [to,  oo),  which  is  Lebesgue  in- 
tegrable  over  every  finite  interval  of  its  domain  and  such  that  |/(t,  x,  u,  d)|  < 
m{t)  in  [to,  oo)  for  every  x  and  d. 

Theorem  2.  Suppose  that  the  admissible  control  laws  are  piecewise  constant  over 
time,  that  disturbances  are  measurable  functions  over  time,  and  that  the  plant  dy¬ 
namics  f  satisfy  Caratheodory  conditions  CC  1-CC  3.  Suppose  also  that 

1.  The  plant  state  space  X  coincides  with  the  Euclidian  space  containing  it 

2.  For  every  u  E  U,  there  is  a  function  L(.)  over  [to,oo),  which  is  Lebesgue  inte- 
grable  over  every  finite  interval  of  its  domain,  and  such  that 

|/(t,x,u,d)  —  /(t,y,u,d)|  <  L{t)  *  |x  —  y|  for  every  x,  y  and  d,  and 
5.  For  every  u,  there  is  a  constant  a  such  that  |/(t,  x,u,d)\  <  a  *  (1  -h  |x|)  for  all  t, 
X  and  d. 

Then  to  every  realizable  play  in  the  continuous  sensing  game  described  above,  there 
corresponds  a  unique  absolutely  continuous  plant  trajectory  defined  over  [t*,oo), 
where  t*  is  the  time  the  play  begins. 


Proof,  We  show  by  induction  on  k  the  uniqueness  and  absolute  continuity  of  the 
plant  trajectory  7  corresponding  to  a  segment  of  a  play  up  to  time  tk  and  defined 
over  the  interval  It  is  sufficient  to  do  the  inductive  step.  Assume  that  the 

statement  is  true  for  k, 

(A)  If  there  is  no  essential  control  move  after  tk,  consider 

y{t)  =  f{t,y{t),Uk,d{t)) 

for  t  >  tfc.  Here  d(.)  is  a  measurable  disturbance  that  occurs  in  the  plant  for  t>tk, 
Uk  is  the  value  of  the  constant  control  function  which  is  part  of  the  essential  Control 
move  at  tk-  We  have  to  show  existence,  uniqueness  of  an  absolute  continuous  function 
satisfying  the  differential  equation  for  t  >  tk  and  beginning  from  the  point  7(4). 
This  would  yield  an  absolutely  continuous  extension  of  the  plant  trajectory  realizing 
the  trajectory  corresponding  to  a  play.  By  assumption  (1),  we  may  assume  that 
j{tk)  G  X.  We  have  to  check  only  that  F{t,  y)  =  /(t,  y,  Uk,  d{t))  satisfies  the  standard 
C2u:atheodory  conditions,  a  uniqueness  of  a  solution  condition,  and  an  extendibility 
condition.  The  standard  Caratheodory  conditions  CC  1-CC  3  are  obtained  from  CC 
1-  CC  3  by  omitting  control  and  disturbance  parameters.  We  will  check  them  for 
F, 

Original  Caratheodory  condition  CC  1:  According  to  CC  1,  there  is  a  set  E  of  the 
measure  0  of  times  such  that  for  any  t  not  in  E,  /(t,  y,  Uk,d)  is  continuous  in  (y,  d). 
Fix  such  a  t.  Then  f{t,y,Uk,d{t))  is  continuous  in  y.  That  is,  F(t,y)  is  continuous 
in  y  for  all  t  not  in  E,  This  verifies  the  original  condition  CC  1. 

Original  CC  2:  We  need  only  show  that  for  every  y,  F(t,  y)  is  measurable  in  t.  By 
CC  2,  /(t,y,ujb,d)  is  measurable  in  (i,d)  for  every  y.  Since  d(.)  is  measurable  and 
the  composition  of  measurable  functions  is  measurable,  it  follows  that  /(t,  y,  Uk,d{t)) 
is  measurable  in  t  for  every  y.  This  is  the  desired  conclusion  for  F. 

Original  CC  3:  The  adapted  CC  3  gives  the  function  m(.)  for  /  depending  on 
u.  So  we  take  the  m  corresponding  to  Uk  and  it  provides  the  desired  bound  for  F. 

Prom  the  standard  Caratheodory  conditions  for  F,  it  follows  that  there  exist 
solutions  of  the  equation  y{t)  =  F(t,y(t))  for  every  t**  >  t*,  x**  e  X  in  some 
interval  [t**,t**  +p]  where  p  >  0  and  y{t**)  —  x**,  see  [12],  page  4. 

The  uniqueness  of  F  easily  follows  from  assumption  (2).  That  is,  choose  the 
function  L(.)  according  to  (2)  which  corresponds  to  Uk-  Then 

\F(t,x)  -  Fit,y)\  =  \f{t,x,Uk,d(t))  -  f{t,y,Uk,d(t))\  <  L{t)  ■  \x-y\. 

It  then  follows  that  there  is  a  unique  trajectory  of  F  passing  through  every  point 
(t**,x**),  see  [12],  page  5. 

Finally  we  consider  the  extension  of  solutions  of  y  =  F(t,y).  Since  the  standard 
Caratheodory  conditions  are  satisfied  by  F,  according  to  [12],  page  7,  every  solu¬ 
tion  can  be  extended  on  both  sides  of  an  initial  condition  to  the  boundary  of  any 
closed  and  bounded  domain  of  F.  By  condition  (3),  choose  a  to  correspond  Uk-  Then 
|^(^5y)l  <  a  •  (1  4-  lyj).  From  estimates  based  on  this  condition,  it  follows  that  the 
states  of  any  trajectory  over  a  finite  closed  interval  of  time  lie  in  a  finite  ball  B  whose 
radius  depends  only  on  size  of  the  interval.  Using  (1),  we  may  choose  for  any  t  >  t*, 
the  domain  for  F  to  be  [t*,t]  x  B,  This  is  a  closed  and  boimded  domain.  Prom  the 
quoted  theorem,  it  follows  that  a  solution  can  be  extended  on  the  whole  of  [t*,t]. 


This  completes  the  demonstration  of  existence  of  the  unique  absolutely  continu¬ 
ous  plant  trajectory  satisfying  y{t)  =  f{t,y{t),Uk^d{t))  for  all  t 

(B)  There  is  an  essential  Control  move  at  tk-^i  >  tk-  For  a  construction  of  a 
plant  trajectory  up  to  tk^i  corresponding  to  a  play,  we  have  to  show  that  there  is 
the  unique  absolutely  continuous  plant  trajectory  satisfying  y{t)  =  /(t,  y{t),Uk,d{t)) 
for  all  t  In  this  case,  the  proof  is  similar  to  the  proof  used  for  case  (A). 

□ 

We  note  that  the  conclusions  of  the  theorem  hold,  in  particular,  for  /  independent 
of  time  and  if  for  all  control  values  w,  /  is  continuous  in  (x,  d)  and  satisfies  conditions 
(l)-(3)  of  the  theorem. 


3.2  Continuous  Input-Discrete  Output  Automata. 

Next  we  want  to  consider  the  analogues  of  an  automaton  winning  strategy  for  contin¬ 
uous  sensing  games.  For  this  purpose,  we  introduce  continuous  input-discrete  output 
automata  to  represent  strategies  for  Control  in  continuous  sensing  games. 

We  adopt  the  following  definition  of  the  behavior  of  an  ordinary  automaton  in 
continuous  time.  First  we  define  the  notion  of  an  automaton  run  on  an  input  word 
as  a  function  of  continuous  time.  Recall  the  ordinary  definition  of  a  run  for  a  finite 
state  automaton.  Let  x  =  xqXi  ...  Xn  be  an  input  word.  Then  a  run  r  =  r(0),  r(l), . . . 
is  the  sequence  of  the  automaton  states  satisfying 

r(0)  =  Sin  and 

r(fc  +  1)  =  M{r{k)^Xk)  for  all  A:  >  0 

where  is  the  automaton  initial  state  and  M  is  its  transition  table. 

Definitions.  Suppose  0  <  to  <  <  —  <  tn  is  an  increasing  sequence  of  times  at 

which  the  letters  of  an  input  word  x  =  xoXi...Xn  are  read.  A  function  r  :  [0,  oo)  5 
is  a  run  of  the  automaton  in  continuous  time  if 

{Sin  iftG[0,to] 

M{r{tk),Xk)  ifn>  k>0Ate  {tk,tk-^i] 

M{r{tn),Xn)  ifte  (tn,00) 

Definition  3  says  that  system  trajectories,  viewed  in  the  automaton  state  space, 
are  functions  of  time  that  are  piecewise  constant,  continuous  from  the  left  at  all 
times. 

Continuously  Reading  Automata 

We  now  introduce  a  definition  of  the  input-output  automata  used  to  model  con¬ 
trollers  capable  of  continuously  reading  input  that  may  continuously  change.  We 
adopt  the  view  that  transitions  are  instantaneous  and  that  state  transitions  are  con¬ 
tinuous  from  the  left  for  automaton  runs  which  correspond  to  a  continuous  stream 
of  input  in  time.  We  also  restrict  attention  to  the  output  of  the  automaton  at  a 
discrete  sequence  of  times. 

Definition  4.  A  continuous-input  discrete-output  automaton  consists  of 
1.  A  nonempty  set  of  states  5, 

2.  A  nonempty  input  alphabet  /, 


3.  An  output  alphabet  J  U  {no  action)  where  J  n  {no  action)  =  0, 

4.  A  transition  table  M  ;  5  x  /  ->  5, 

5.  An  output  function  J?  :  5  x  /  ^  J  U  {no  action)^ 

6.  An  initial  state  8in> 

For  example,  the  controller  described  previously  for  water  level  translates  into 
the  following  continuous  input-discrete  output  automaton.  Its  set  of  states  is  5  = 
{son,5o//},  the  initial  state  is  Sin  =  son,  the  input  alphabet  is  I  =  {y  :  y  >0,y  £ 
R),  and  the  alphabet  of  essential  outputs  is  J  =  {pon,poff).  Here  R  is  the  set  of 
reals.  The  transition  table  and  the  output  function  are  defined  as  follows: 

'  [son  ify<g 

nff  \  {son  \iy<h 

TTfcyr,  if  y  >  5 

^  I  no  action  if  y  <  g 

»(»»//.»)= ''/fi. 

\  J  action  if  y  >  n 

Definition  5.  Let  TIME  =  [^*,  oo),  r(0)  =  Sin.  Suppose  that  x(.)  maps  TIME 
into  /.  A  run  of  a  continuous-input  discrete-output  automaton  corresponding  to  the 
input  stream  x(.)  is  a  function  r  :  TIME  S  such  that  for  every  t  G  TIME, 
there  is  a  duration  r  >  0  such  that  r(t')  =  M{s,x{t))  for  every  f  €  {t,t-\-  r].  The 
output  function  of  a  continuous-input  discrete-output  automaton  corresponding  to 
the  input  stream  x(.)  and  a  run  r  is  the  function  h  :  TIME  ^  J  U  {no  action} 
defined  by  h{t)  =  H{r{t),x{t)). 

Given  a  run  r  of  a  continuous-input  discrete-output  automaton  A  corresponding 
to  an  input  stream  x(.),  we  define  the  set  of  switching  times  of  r,  SW{r),  to  be  the 
set  of  all  t  €  TIME,  such  that  M(r(t),x(t))  ^  r(t). 

The  difference  between  our  continuous  input-discrete  output  automaton  and  a 
standard  Mealy  machine  is  that  we  allow  the  input  to  be  an  arbitrary  function  of 
continuous  time  rather  than  a  piecewise  constant  function  of  time  which  refiects  input 
at  discrete  instants  only.  We  call  the  subset  J  of  the  automaton  output  alphabet, 
the  alphabet  of  essential  outputs. 

The  definition  of  run  for  a  continuous-input  discrete-output  automaton  gives 
rise  to  piecewise  constant  and  continuous  from  the  left  state  space  functions  which 
represent  the  transitions.  One  of  the  reasons  we  adopt  this  definition  is  to  avoid 
the  difficulties  associated  with  the  following  automaton.  Let  S  =  {0, 1},  Sin  =  0, 
J  =  [5, 10],  ^0  =  1)  and  the  automaton  transition  function  be  given  by 

M(0,5)  =  l 
M(l,x)  =  0  for  X  >  5. 


Suppose  the  input  function  is  x(t)  =  5  •  t.  Then  a  transition  should  occur  at 
t  =  to.  However  if  such  a  transition  does  occur,  then  at  any  later  instant  ti  >  to 
where  the  new  state  is  s  =  1  another  transition  from  the  state  s  =  1  back  to  the 
state  s  =  0  must  occur.  So  at  some  t2  >  ti,  the  automaton  is  again  in  state  s  =  0. 
This  would  imply  that  such  transition  times  occur  arbitrarily  close  to  to-  But  this  is 
inconsistent  with  our  intuition  of  an  automaton  transition  while  continuously  reading 
the  input  because  there  is  no  finite  interval  of  the  form  (to,  to  +  t]  during  which  the 
automaton  is  in  a  fixed  state. 

Definition  6.  Assume  that  the  automaton  input  alphabet  /  is  a  subset  of  a  Eu¬ 
clidean  space.  Call  the  sets  =  {i  :  M{s,i)  ^  s}  the  switching  sets.  We  say  that 
a  continuous  input-discrete  output  automaton  A  has  separated  switching  sets  if 
for  every  s,  and  s',  the  Euclidean  distance  between  the  sets  Gg  and  Gg'  is  positive, 
i.e.  p{Gg,Ggf)  >  0.  Here  piGg,Ggf)  =  inf{p{x,x^) :  x  €  Gg,  x'  G  Ggt}  where  p{x,x^) 
is  the  usual  Euclidean  distance  function. 

Theorem  7.  Consider  a  continuous  input  automaton  with  the  following  properties. 

(a)  Its  set  of  states  is  finite. 

(b)  Its  input  alphabet  is  a  subset  of  a  Euclidean  space  E. 

(c)  Its  switching  sets  of  inputs  are  separated. 

(d)  Its  switching  sets  of  inputs  are  closed  in  the  subset  topology  of  E. 

Then  for  every  input  function  x(.)  which  is  continuous  over  TIME  =  [t*,  oo),  there 
is  o  unique  run  of  the  automaton  over  a;(.).  Moreover,  the  set  SW{r)  of  switching 
times  during  the  run  is  discrete  with  no  limit  points  in  TIME. 

Proof  Clearly,  if  does  not  intersect  the  range  of  x{.),  then  r(t)  =  Sin  for  all 
t  €  TIME.  In  this  case,  SW  is  empty  and  clearly  the  conclusions  of  the  theorem  are 
satisfied.  Suppose  Gg^^  does  intersect  the  range  of  a:(.)  .  Then  ^  0.  This 

set  is  also  closed,  since  by  assumption  a;(.)  is  continuous  and  Gg.^  is  closed.  Hence, 
there  is  the  least  time  to  such  that  to  €  x~~^{Ggi^).  This  is  the  first  switching  time. 
We  include  to  in  SW.  We  associate  the  state  So  =  Sm  with  to- 

Next  suppose  we  have  constructed  an  increasing  sequence  of  switching  times 
to,...,tjfe  and  the  sequence  of  the  corresponding  states  up  to  so,...,Sk  at  these 
switching  times.  Consider  Sk^i  =  M{skyx{tk))*  Then  either  fl  (t*,  oo)  = 

0,  in  which  case  r(t)  =  sjk+i  for  all  t  >  tjb,ora:’’^(G,j^^i)n(tjb,oo)  7^  0.  If 
(tA.,00)  ^  0,  then  the  set  H  [tjbjoo)  is  closed.  Moreover  tk  cannot  be  a 

limit  point  of  00).  That  is,  if  tk  were  such  a  limit  point,  there  would 

be  a  sequence  of  points  t'-  in  converging  to  tk*  But  then,  because  of  the 

continuity  of  x(.),  it  must  be  that  is  a  limit  point  of  a  sequence  x(t'  )  of  points 
from  Gg^^^ .  This  would  contradict  the  separateness  of  from  the  switching  set 
Gg^  containing  x(ffc).  Then  we  let  tk-^i  be  the  least  element  of  fl  [tk,  00). 

Thus  by  induction,  we  can  define  two  sequences  {tk}  and  {s^}  such  that  for  all 

k, 


tk+i  e  X  n  {tk,  00) 


and 


Sk+i  =  M{sk,x{tk)). 

Let  SW  be  the  set  of  elements  in  the  first  sequence.  We  claim  that  SW  has  no  finite 
limit  points  in  TIME.  Indeed  suppose  the  sequence  tk  converges  to  i**  >  0.  Since 
the  set  of  states  5  is  finite,  there  is  a  strictly  positive  number  a  =  min{p{G8yGs‘)  : 
8  ^  s'jSjs'  €  S}.  Choose  e  <  a/2.  By  continuity  of  x,  there  exists  an  J  >  0  such 
that  |t <  6  implies  \x{t)  —  x{t'^)\  <  e.  Consider  A:o  such  that  for  every  k  >  ko, 
|4+i  -tk\<S.  Then  for  all  such  k 

\x{tk^i)  -  x(tk)\  <2-6  =  a 

However  for  all  fc,  x{tk)  €  Gsf,  and  by  the  separateness  of  the  switching  sets,  it 
follows  that 

\x{tk+i)  -  x{tk)\  >  a. 

This  is  a  contradiction  and  hence  the  set  SW{r)  has  no  finite  limit  points. 

Since  no  transitions  are  possible  at  times  between  switching  times,  we  have, 
besides  the  constant  run  mentioned  above,  two  more  types  of  runs  depending  on 
whether  the  set  of  switching  times  SW  is  finite  or  infinite.  If  SW  is  finite  with  last 
switching  time  tnj  then  set 

{Sjn  if  t  =  to 

M{r{tk),x{tk))  iin>  k>0Ate 
M(r(tn),a:(tn))  if  f  €  (4,oo). 

If  SW  is  infinite,  then  set 

/.\  _  f  ^in  if  t  =  io 

^  ^  ”  \M{r{tk),x{tk))  ifk>0Ate 

The  uniqueness  of  runs  follows  by  induction  on  the  switching  times. 

□ 

Next,  we  single  out  a  property  of  a  continuous-input  discrete-output  automaton 
which  has  been  proved  in  the  previous  proposition,  but  can  be  established  with  a 
slightly  weaker  assumption.  We  will  use  this  fact  later  in  this  section. 

Propositions.  Suppose  that  the  premises  of  Theorem  7  hold,  but  the  requirement 
that  the  set  of  the  automaton  states  be  finite  is  omitted.  Then  at  any  state  at  which 
the  automaton  is  continuously  reading  a  continuous  function  x  :  TIME  -y  /,  either 
the  automaton  remains  in  this  state  forever  or  there  is  a  finite  time  t>  0  at  which 
a  transition  to  a  different  automaton  state  takes  place. 

Definition  9.  A  run  r  of  a  continuous-input  discrete-output  automaton  is  realiz¬ 
able  if 


(a)  Both  transitions  to  new  states  and  essential  outputs,  occur  only  at  discrete 
times  DT  =  {to  <  ti,<  ...}.  That  is,  M(r(t),x(t))  =  r(t)  and  H{r{t),x{t))  = 
no  action  for  t  not  in  DT  and  r{tk)  i=‘  r{tk^i)  and  ff(r(fifc),  x(t^))  G  J  for  A;  >  0. 


(b)  r{t)  =  M{r{tk),x{tk))  for  every  k>0,te  (tk,tk+i]. 

(c)  inf{{{tk+i  -tk):k>  0})  >  0. 


If  the  sequence  DT  is  finite  and  n  is  the  last  index  k  occurring  in  it,  define  in+i  =  oo. 
Then  (a)-(c)  of  the  definition  apply  to  the  interval  (^n,^n+i)- 

For  example,  the  continuous  input-discrete  output  automaton  representing  the 
controller  for  the  water  pump  given  above  satisfies  the  premises  of  the  Proposition 
7.  Therefore  it  has  runs  over  continuous  water  level  trajectories  y{.).  Moreover,  the 
proposition  tells  us  that  these  runs  are  realizable. 

Conditions  (a)  and  (b)  given  in  the  definition  of  a  realizable  run  refiect  our 
intuition  of  automaton  transitions  as  described  above.  The  definition  synchronizes 
transitions  to  new  states  with  essential  outputs.  For  automata  with  separated  switch¬ 
ing  sets  which  satisfy  the  other  conditions  of  Theorem  7,  once  this  synchronization 
is  present,  every  run  is  automatically  realizable. 

Condition  (c)  prevents  the  set  of  transition  times  DT  from  having  finite  limit 
points.  We  call  (c)  a  realizability  condition.  The  definition  of  realizable  run  here  has 
the  same  motivation  as  that  of  realizable  time  sequence  in  [24]  and  of  “bounded 
from  below”  sampling  intervals  for  a  controller  in  [36]. 

By  taking  J  =  X  x  [0,  oo),  we  can  make  the  automaton  transition  table  and  the 
output  function  depend  explicitly  on  time,  M(s,  x,  t);  if(s,  x,  t).  We  can  then  ensure 
that  the  conditions  (a)-(c)  are  satisfied  for  all  runs  of  the  automaton  over  any  input 
function  x(.)  by  choosing  a  discrete  set  DT  =  {to  <  <  . . .}  satisfying  (c),  defining 

M(s,x,t)  =  A(s,x)  and  jff(s,x,t)  =  B(s,x)  for  t  €  DT,  and  defining  M(s,x,t)  =  s 
and  H{s,  x,  t)  =  no  action  for  t  ^  DT,  where  A  and  B  are  transition  tables  which 
are  not  dependent  on  time. 


3.3  Automata  as  Strategies 

Next  we  explain  how  we  can  use  a  continuous-input  discrete-output  automaton  as  a 
strategy  for  Control  in  our  continuous  sensing  games.  First  choose  an  input  alphabet 
I  =  X  and  an  alphabet  of  essential  outputs  J  =  (7  x  [0,  oo).  Control  uses  the 
continuous-input  discrete-output  automaton  in  the  following  way.  Suppose  t  is  the 
current  time  and  tk  <  t  is  the  last  time  the  automaton  output  was  an  essential 
Control  move.  If  y{t)  is  the  current  input,  the  automaton  stays  in  its  current  state 
s  or  shifts  into  another  state  according  to  its  transition  function  M{s,y{t))  and 
outputs  the  respective  Control  move  according  to  its  output  function  H{s,y{t)).  If 
at  time  t,  there  is  a  shift  to  another  state,  then  the  next  essential  Control  move 
occurs  at  t  and  =  t.  However  the  sequence  of  automaton  states  resulting  may 
not  form  a  run,  much  less  a  realizable  run,  in  the  course  of  reading  an  input.  We 
call  the  automaton  a  realizable  strategy  for  Control  if  whenever  Control  uses  the 
automaton  as  its  strategy,  then  the  resulting  play  produces  a  realizable  run. 

Since  the  set  of  plant  states  is  usually  a  subset  of  a  Euclidean  space,  it  is  natural  to 
consider  automata  with  closed  and  separated  switching  sets  as  strategies  for  Control. 
However  even  if  Control  use  this  type  of  automaton,  it  will  not  always  produce  plays 
whose  realizability  can  not  be  established  by  appealing  to  Theorem  7.  The  reason  is 
that  the  automaton  output  affects  the  future  input  and  may  result  in  the  automaton 


input  not  being  a  continuous  function  of  time,  so  that  Theorem  7  does  not  apply. 
Such  an  automaton  is  given  below. 

Control  strategies  need  not  produce  realizable  runs 
Consider  the  plant  with  a  scalar  control  and  disturbance: 

y  =  d,  for  u  =  0 
y  —  -d,  for  u  =  1 
de  Z,  d  >  0 

y(0)  —  0,  and  initially  the  control  parameter  is  set  to  0. 

Consider  the  following  automaton  represented  strategy  for  the  player  Control: 


5  =  {0,1} 

Sjn  =  0 

I  ={y:yeR} 

J  =  {u  :=  0,  u  :=  1}.  (Here  we  think  of  the  essential  control  moves  as  orders 
to  set  the  control  parameter  to  the  indicated  values.) 

Let  a  >  ^  >  0  be  given  and  let  the  transition  table  be  defined  by: 


M(0,y)  =  | 


1  if  y  >  a 
0  if  y  <  a 

0  if  y  <  /3 
liiy  <  0 


Here  we  assume  The  output  function  is  defined  by 


ff(i,y)  =  { 


u  :=  1  if  y  > 
no  action  if  y  < 

u  :=  0  if  y  > 
no  action  if  y  < 


a 

a 

0 

0 


It  is  easy  to  see  that  there  are  exactly  two  switching  sets,  namely.  Go  =  (a,  oo) 
and  Gi  =  (— oo,^).  Thus  since  0  <  a,  these  are  separated  switching  sets.  If  this 
strategy  always  produced  realizable  runs,  then  the  corresponding  plant  trajectories 
would  be  continuous.  But  we  exhibit  a  plant  trajectory  from  a  game  which  uses 
the  automaton  as  a  Control  strategy  and  which  produces  a  discontinuous  plant 
trajectory.  Assume  that  the  disturbance  is  initially  d  =  1  at  time  i  =  0,  and  that 
the  disturbance  doubles  after  each  automaton  state  switch.  We  will  get  the  first 
switch  at  time  a,  the  second  will  occur  (a  —  0)/2  seconds  later,  the  third  will  occur 
(a  —  0)/i^  after  the  second,  and  so  on.  The  switch  times  tk  are  the  sums  of  the  first 
k  terms  of  this  series.  That  is,  they  are  tk  =  YliZo  sequence  has  a  finite 

limit  point  2a,  Hence  there  are  times  arbitrary  close  to  2a  from  the  left  where  the 
plant  state  is  y  =  a  and  the  plant  state  y  =  0,  Thus  the  plant  trajectory  is  not 
continuous  at  i  =  2a, 

Plants  with  Realizable  Control  Strategies 

Next  we  define  a  class  of  plants  together  with  a  class  of  input-output  automata 
which  are  guaranteed  to  produce  realizable  strategies  for  Control. 


Suppose  the  plant  is  modeled  by  a  system  of  differential  equations  of  the  form 
3?  —  G  XJ^d  G  DyX  G  Xj 

where  u  is  a  control  parameter  and  d  is  disturbance  parameter.  Assume  that  U  C 
E^,  D  C  E*,  and  that  X  C  E”.  We  allow  an  additional  source  of  nondeterminism  in 
the  plant  of  the  following  sort.  For  each  pair  of  parameters  (u,  u'),  there  is  a  delay  in 
resetting  u  to  w',  bounded  by  distance  p{u,  u').  Let  t*  and  x*  be  the  initial  conditions 
for  the  plant.  Assume  that  we  have  only  piecewise  constant  control  functions,  so  that 
we  can  identify  control  parameter  value  u  with  the  constant  control  function  u{t)  =  u 
for  alH  >  t',  where  f  is  a  resetting  time  of  a  previous  control  parameter  value  to  u. 

Consider  the  following  strategy  A  for  Control  in  the  continuous  sensing  game 
in  which  the  plant  state  x  is  being  sensed  by  Control.  Let  A  be  a  continuous-input 
discrete-output  automaton  such  that: 

1.  Its  state  space  S  is  finite. 

2.  Its  input  alphabet  is  /  =  X. 

3.  Its  alphabet  of  essential  control  moves  is  J  =  (7  x  [0,oo). 

4.  Its  transition  table  M  satisfies  the  condition  of  separateness  of  switching  sets. 

5.  The  switching  set  for  the  initial  state  contains  the  initial  plant  state  x*  G  Gsi^  • 

6.  The  automaton  output  function  E(s,  x)  produces  an  essential  output  only  when 

X  is  in  the  switching  set  Gg,  otherwise  ff(s,x)  =  no  action. 

In  particular,  H{sinjX*)  =  for  some  u  eU  so  that  the  first  output  to  be 

produced  by  the  automaton  is  an  essential  move. 

Theorem  10.  Suppose  that  the  plant  is  modeled  hy 

y  =  fit, y,u,d) 

as  described  above.  Suppose  that  f  satisfies  the  Caratheodory  conditions  CC  1,  CC 
2,  and  CC  3,  where  we  assume  that  there  is  a  fixed  function  m(.)  for  /,  inde¬ 
pendent  of  the  value  u  of  the  control  parameter  for  condition  CC3.  Assume  that 
the  automaton  A  described  above  has  closed  separated  switching  sets.  Then  for  every 
play  /i  consistent  with  Control  following  the  strategy  A,  the  resulting  automaton  state 
function  is  a  realizable  run  of  A, 

Proof,  We  show  first  that  the  use  of  A  by  the  player  Control  results  in  the 
production  of  runs  of  A,  By  the  conditions  in  the  paragraph  preceding  the  theorem, 
the  first  control  move  according  to  A  is  essential.  The  control  laws  here  are  constant 
functions.  Suppose  there  are  no  more  essential  control  moves.  Since  the  control  laws 
here  are  constant  functions,  an  essential  move  creates  continuous  plant  trajectories 
because  of  the  three  Caratheodory  conditions  satisfied  by  the  plant  model.  It  then 
follows  by  Theorem  10  that  the  corresponding  run  is  realizable. 

Now  suppose  a  finite  number  of  essential  control  moves  were  made  from  the 
beginning  of  the  play.  Consider  a  time  at  which  the  last  essential  control  move 
was  made.  That  is,  if  s*  is  the  state  of  A  at  time  i**,  then  the  automaton  input  x** 
at  time  t**  is  in  the  switching  set  G,* .  We  may  assume  that  the  interaction  of  the 
automaton  A  and  the  plant  has  produced  the  plant  trajectory  7  up  to  time  t**  and 


=  X**.  In  other  words  E  We  wish  to  show  that  there  is  a  positive 
r  >  0  such  that  there  is  no  essential  automaton  outputs  and  transitions  to  new 
states  in  the  interval  (t**,t**  +  r).  It  is  easy  to  see  that,  even  with  the  finite  delay 
corresponding  to  resetting  the  control  according  to  =  H{s*,x**)j  the  input 

to  the  automaton  is  a  continuous  plant  trajectory  because  the  function  /  satisfies 
the  Caratheodory  conditions  CC  1,  CC  2,  and  CC  3.  It  follows  from  Proposition 
4.3  that  there  are  two  alternatives.  The  first  alternative  is  that  the  automaton  stays 
forever  in  this  state.  This  produces  a  valid  realizable  run  and  play  with  t**  being 
the  last  essential  Control  move.  The  second  alternative  is  that  there  is  r  >  0  such 
that  the  next  essential  move  occurs  at  +  r.  this  means  that  there  are  no  essential 
outputs  or  transitions  to  new  states  in  (t**,t**  +  r),  since  these  occur  at  the  same 
time  according  to  the  definition  of  A,  and  the  transitions  to  new  states  do  not  occur 
in  this  alternative  for  such  an  interval. 

It  follows  that  when  Control  uses  A  in  the  game,  the  set  of  times  at  which 
essential  moves  are  made  is  a  discrete  set.  We  let  -DT(/x)  denote  this  set  for  the  play 
/i.  Next  fix  a  play  /z  and  consider  an  initial  sequence  {tk  :  k  >  0}  of  DT  =  DT{fjL) 
that  begins  with  the  time  of  the  first  essential  control  move.  Clearly,  every  finite 
initial  segment  of  this  sequence  determines  a  position  in  the  game  for  which  there 
is  a  corresponding  continuous  plant  trajectory.  Suppose  that  ^  is  a  limit  point  of 
this  sequence.  Due  to  assumption  CC  3,  we  get  the  following  estimate  for  any  plant 
trajectory  7: 

|7(t')  -  7(i)|  <  ^  m{t)dt. 

Since  the  integral  is  absolutely  continuous,  it  follows  that  the 

lirrik-^oobitk+i)  - 'ritk)\  =  0. 

By  the  argument  above,  it  follows  that  for  every  k,  'y{tk)  €  (7*^,  where  s*  is  the 
automaton  state  at  which  the  transition  occurs  at  the  switch  time  i*.  Moreover,  the 
sets  Gsu+i  distinct  since  the  states  Sk-\-i  and  Sk  are  distinct.  The  fact 

that  the  above  limit  is  0  and  the  fact  that  there  are  only  a  finite  number  of  switching 
sets  would  imply  that  for  some  pair  of  states  s  and  s',  the  distance  between  Gg  and 
Gs>  is  0.  But  this  contradicts  our  assumption  that  there  is  nonzero  distance  between 
the  switching  sets.  Thus  the  sequence  {tk}  has  no  finite  limit  points.  Since  the  set 
of  switching  instants  DT  is  the  set  of  times  of  essential  Control  moves  and  this  set 
has  no  finite  limit  points,  the  corresponding  play  is  realizable.  □ 

We  note  that  for  the  water  level  monitor  problem  describe  above,  the  bound 
function  for  the  Caratheodory  condition  CC  3*  is  m(^)  =  max{a,b}  •  t  Thus  the 
plays  produced  by  the  suggested  automaton  controller  are  all  realizable. 

4  From  Continuous  to  Discrete  Sensing  Games 

In  this  section  we  fiilly  analyze  the  water  pump  example.  We  start  by  explicitly 
constructing  a  continuous-input  discrete-output  automaton  A{g,  h)  for  a  pair  of  pa¬ 
rameters  ^  >  fc  as  described  in  the  previous  section.  By  Theorem  10,  we  know  that 
if  Control  uses  A{g,  h)  for  its  strategy  in  the  continuous  sensing  game  for  the  water 


level  monitor,  then  it  will  always  produce  realizable  runs  for  A{g^  ft).  We  shall  show 
that  for  any  desired  water  levels,  u  <  u,  we  can  pick  (^,  h)  in  such  a  way  that  if 
Control  uses  the  automata  A{g^  h)  for  its  strategy  in  the  continuous  sensing  game, 
then  Control  will  win  in  the  sense  that  we  will  guarantee  that  at  all  times  t,  the 
water  level  y{t)  will  satisfy  u  <  y{i)  <  v  assuming  that  u  <  y(0)  <  v.  Then  we 
shall  show  how  we  can  use  the  continuous-input  discrete-output  automaton  A{g^  h) 
to  design  a  finite  automaton  which  will  control  the  plant,  that  is,  the  water  tank 
plus  pump,  to  meet  the  desired  performance  specification.  Finally,  we  shall  show 
that  we  can  explicitly  extract  Kohn-Nerode  small  topologies  which  will  verify  the 
controllability  and  observability  of  our  discrete  control  strategy. 

The  (g,h)-Automaton  A{g^  h) 

With  any  pair  of  positive  numbers  with  g  >  h  we  associate  a  (g^h)- 

automaton  with  continuous  input  alphabet  and  a  three  letter  output  al¬ 

phabet.  (This  is  essentially  the  same  automaton  that  was  described  in  the  previous 
section.) 

1.  The  input  alphabet  consists  of  the  numbers  in  interval  M  of  possible  water  levels 

y- 

2.  The  two  automaton  states  are  son,soff. 

3.  The  three  letter  output  alphabet  is  pon,poff,no  action. 

The  transition  table  and  output  function  of  this  automaton  are  defined  as  follows. 

^  (no  action  iiy  <g 

uf  ^  _  I  if  y  < 

(50//,  y)  I  action  if  y  >  h 

The  the  switching  sets  for  A{gyh)  are  Ggon  =  [9^00)  and  Gaoff  =  (— oo,h]. 
We  can  thus  guarantee  that  A{g^  h)  has  separated  switching  sets  if  we  impose  the 
requirement  that  g  >  h. 

Theorem  11.  If  parameters  g  and  h  in  the  continuous  sensing  game  for  the  water 
level  monitor  with  maximum  delay  d  satisfy  the  conditions: 

(1)  g  >  hy  (2)  g  +  a  •  d  <  V,  (3)  h  —  b  •  d  >  Uy 

(4)  h-b-d<  2/(0)  <  gy  (5)  >  d,  (6)  >  d, 

then  the  strategy  A{gy  h)  is  a  winning  strategy  for  the  player  Control  in 
where  the  initial  state  of  Control  is  son  and  the  initial  state  of  the  pump 


(2) 

any  game 
is  pan. 


Proof  Suppose  /x  =  Iq,  zq,  Fi,  2:1, ...  is  a  play  consistent  with  A,  We  have  to  show 
two  things.  First  we  must  show  that  the  strategy  induced  by  A{g,  h)  is  applicable  at 
every  position  of  Control  in  this  play.  That  is,  we  must  show  that  if  Control  using 
this  strategy,  then  he  never  gets  stuck  in  the  sense  that  he  is  unable  to  make  a  move 
according  to  the  strategy.  This  is  the  perpetual  property  [48],  [38].  Second  we  must 
show  that  A(g,  h)  induces  a  winning  strategy  for  Control,  i.e.  that  the  water  level 
trajectory  j/(.)  corresponding  to  any  play  consistent  with  the  game  initial  condition 
and  the  strategy  i4(^,  h)  has  the  property  that  for  all  times  t  >  0,  u  <  y{t)  <  u.  We 
show  both  properties  by  induction  on  the  length  of  a  position  in  the  play. 

The  initial  position  of  the  play  is  p  =  {pon,0)  and  the  initial  trajectory  of  the 
plant  is  just  (0,y(0)).  That  is,  the  initial  control  sent  to  the  plant  is  that  the  pump 
should  be  on.  Now  since  y{Q)  satisfies  v  <h  —  b*  d  <  y{0)  <  g  <  v,  we  see  that  the 
initial  trajectory  is  within  acceptable  bounds. 

Consider  the  first  block  of  plant  moves  which  is  specified  by  its  corresponding 
trajectory  Yq.  By  our  assumptions,  we  have  that  for  all  t, 

0  <  a'  <  Yo{t)  <  a. 

Thus  Yq  will  be  a  strictly  increasing  function  so  that  there  will  be  some  time  >  0 
such  that  io(^i)  =  9-  It  is  easy  to  see  that  h  <{g  -  y(0))/a' 

Thus  at  time  ti.  Control  issues  the  order  that  the  pump  should  be  turned  off 
and  switches  to  state  soff.  Thus  zq  =  (po//,ti}. 

Now  consider  the  next  block  of  plant  moves  which  is  specified  by  its  corresponding 
trajectory  Fi.  Because  of  the  delay  in  switching  from  the  pump  being  on  to  the  pump 
being  off  after  the  control  order  to  turn  the  pump  off  has  been  issued,  there  is  some 
0  <  Ti  <  d  such  that  the  pump  remains  on  between  time  ti  and  time  ti  -f  ri  and 
then  the  pump  turns  off.  Thus  the  corresponding  trajectory  Yi  satisfies 

0  <a*  <Yi{t)  <a 
0  >  —V  >  Yi{t)  >  -6  if  t  >  +  Ti 

It  is  then  easy  to  see  that  the  trajectory  Yi  must  reach  its  maximum  at  time 
t  =  +  Ti  and  that  this  maximum  value  is  bounded  by  ^  +  ari  <  g  +  o.d  <  v. 

After  time  H-  n,  Y\  is  strictly  decreasing  so  that  there  must  be  some  time  *2  > 
such  that  ^1(^2)  =  h.  It  is  easy  to  see  that  d  <  <  ^2  -  Since 

t2  ~  >  d  it  follows  that  the  state  the  pump  will  be  soff  at  time  ^2*  Thus  at 

time  ^2)  Control  issues  a  order  that  the  pump  be  turned  on  and  switches  to  state 
son.  Thus  zi  =  (pon,i2)*  It  then  easily  follows  that  the  values  of  the  trajectory  Yi 
between  times  t  =  ti  and  t  —  takes  on  its  maximum  value  at  time  ti  -1-  ri  and  its 
minimum  value  at  time  t2  where  ^1(^2)  =  h.  Thus  the  values  of  yi(^)  lie  between  h 
and  g-\-ad  and  hence  meets  our  performance  specifications. 

Now  consider  the  next  block  of  plant  moves  which  is  specified  by  its  corresponding 
trajectory  F2.  Again,  because  of  the  delay  in  switching  from  the  pump  being  off  to 
the  pump  being  on  after  the  control  order  to  turn  the  pump  on  has  been  issued, 
there  is  some  0  <  r2  <  d  such  that  the  pump  remains  off  between  time  t2  and  time 
t2  H“  T2  and  then  the  pump  turns  on.  Thus  the  corresponding  trajectory  I2  satisfies 

0  >  -6'  >  F2W  >-b  ift2<t<t2+r2 
0  <  a'  <  Y2{t)  <a  iit>  t2-^T2 


It  is  then  easy  to  see  that  the  trajectory  I2  must  reach  its  minimum  at  time  t  =  t2‘fr2 
and  that  this  minimum  value  is  bounded  below  by  /i  —  or2  '>h  —  hd>u.  After  time 
*2  +  72,  Y2  is  strictly  increasing  so  that  there  must  be  some  time  tz  >  t2  such  that 
¥2(13)  =  g.  It  is  easy  to  see  that  d  <  Since  t3-t2>dit 

follows  that  the  state  the  pump  will  be  son  at  time  ^3.  At  time  ts,  Control  issues  a 
order  that  the  pump  be  turned  off  and  switches  to  state  so//.  Thus  Z2  =  (po/ /,  tz). 
It  then  easily  follows  that  the  values  of  the  trajectory  ¥2  between  times  t  =  *2  and 
t  =  tz  takes  on  its  minimum  value  at  time  t2  -h  T2  and  its  maximum  value  at  time  *3 
where  >2(^3)  =  9-  Thus  the  values  of  l2(^)  lie  between  h  —  bd  and  g  and  hence  meet 
our  performance  specifications. 

Thus  the  behavior  of  the  system  between  the  position  ending  in  zi  and  the 
position  ending  in  zz  meets  the  performance  specification  and  the  requirement  that 
Control  can  follow  the  strategy  determined  by  A{g,h),  Note  that  at  time  tj,  the 
water  level  is  g  and  the  pump  is  on  and  at  time  tz  the  water  and  the  pump  is  on. 
It  is  then  straightforward  to  prove  by  induction  that  at  time  t2n+i  the  water  level 
will  be  g  and  the  pump  will  be  on  and  that  exactly  the  same  analysis  will  apply  to 
the  behavior  of  the  system  between  the  position  ending  in  Z2n+i  and  the  position 
ending  in  ^2n+3-  Hence  it  follows  that  the  strategy  for  Control  induced  by  A(g,  h)  is 
a  winning  strategy  for  Control  as  claimed.  □ 

It  should  be  clear  that  in  the  statement  of  Theorem  11  we  can  replace  the  as- 
sumption  that  the  pump  is  initially  on  and  u  <  2/(0)  <  ghy  the  assumption  that  the 
pump  is  initially  off  and  h  <  2/(0)  <  ^  4*  o  •  d  and  the  conclusion  of  the  Proposition 
will  continue  to  hold. 


4.1  The  (y,  ft) —Automata  for  Discrete  Sampling  and  Measurement 
Errors 

We  now  modify  our  continuous  sensing  game  for  the  water  level  monitor  in  two  ways. 
First  we  shall  assume  that  Control,  instead  of  continuously  sensing  the  plant  state, 
senses  the  plants  state  only  at  discrete  times  to  <  <  ^2  <  •  •  •»  where  there  is  some 

positive  ^  >  0  such  that  -  tfc  >  4  for  all  A:  >  0. 

Second,  we  shall  assume  that  Control  is  not  able  to  exactly  measure  the  plant 
state,  but  only  that  Control  can  measure  the  plant  state  within  some  error  e.  Our 
goal  is  to  specify  a  continuous-input  discrete-ouput  automaton  strategy  for  Control 
in  such  a  game  and  the  sequence  of  sampling  times  to  <  <  ^2  <  •  •  •  so  that  if 

Control  measures  the  plants  state  at  the  times  to  <  ^1  <  ^2  <  • « •  with  an  error  of 
no  more  than  e  and  follows  the  strategy  induced  by  the  continuous-input  discrete- 
ouput  automaton,  then  Control  will  ensure  that  the  plant  meets  the  performance 
specifications. 

In  this  case,  we  shall  assume  that  to  =  0  and  that  tk  =  kA  for  all  A:  >  0  so 
that  we  axe  sampling  every  A  seconds,  where  A  >  d  and  d  is  the  maximum  delay 
that  can  occur  between  the  time  at  which  Control  issues  a  order  to  the  pump  to 
turn  off  or  on  and  the  time  the  pump  actually  achieves  the  state  required  by  the 
order.  Moreover,  we  shall  continue  to  use  the  automaton  A{g,  h)  for  the  strategy  for 
Control.  Thus  the  behavior  of  the  system  is  the  following: 

A.  Suppose  that  the  automaton  is  in  state  son  and  receives  as  input  measurement 
m.  Then,  instantaneously, 


1.  if  m  >  ^,  then  the  automaton  outputs  poff  and  also  shifts  its  state  to  soff, 

and 

2.  if  m  <  g,  then  the  automaton  remains  in  state  port,  and  outputs  no  action. 

B.  Suppose  that  the  automaton  is  in  state  soff  and  and  receives  input  measurement 
m.  Then,  instantaneously, 

1.  if  m  <  /i,  then  the  automaton  outputs  pon  and  shifts  to  state  son,  and 

2.  if  y  >  h,  then  the  automaton  remains  in  state  soff  and  outputs  no  action. 

Thus  our  problem  is  find  A  and  the  parameter  g  and  h  to  ensure  that  the  water 
level  y{t)  stays  within  the  desired  bounds,  i.e.  that  for  all  t,  u  <  y{t)  <  v.  First  of 
all,  since  we  pick  A>  d,we  will  be  guaranteed  that  the  plant  and  automaton  states 
correspond  to  each  other  at  the  end  of  each  sampling  interval.  That  is,  if  initially 
the  plant  state  and  the  initial  state  of  A{g,  h)  are  such  that  if  the  initial  state  of 
A{g,  h)  is  soff,  then  the  pump  is  off  and  if  initial  state  of  A{g,  h)  is  son,  then  the 
pump  is  on,  then  at  some  time  before  the  end  of  each  sampling  interval  the  state  of 
A{g,  h)  the  pump  will  correspond  to  each  other. 

It  is  then  quite  easy  to  derive  the  necessary  conditions  on  the  parameters  g  and 
h  to  guarantee  that  the  control  automaton  A{g,  h)  provides  a  winning  strategy  for 
Control  in  our  modified  game.  That  is,  all  we  do  have  to  do  is  analyze  the  plant 
trajectories  for  given  input  measurement  and  states  of  A{g,h).  We  consider  the 
following  cases. 

Case  1  Suppose  that  the  plants  state  is  son  and  at  time  t*,  Control  receives 
a  measurement  ruk  <  y.  Now  by  assumption,  if  the  actual  water  level  at  time  tk  is 
y{tk),  then 

mk-e<  y{tk)  <  mib  -f  e. 

Assume  also  that  the  pump  is  on  at  time  4  so  that  in  this  case  the  automaton 
remains  in  state  son  and  issues  the  order  no  action  and  the  pump  remains  on  for 
the  next  A  seconds.  Then  since  the  plant  trajectory  y(.)  between  tk  and  tk-\-i  =tk-l-A 
must  satisfy 

0  <  a'  <  y{t)  <  a, 

it  is  easy  to  see  that  y{t)  is  a  strictly  increasing  function  in  this  interval  and  that 

<  y{h)  +  aA<Tnk-\’aA  +  e<g-\‘aA  +  e. 

Now  if  we  find  that  the  measurement  received  at  time  tk-^i,  mk+i,  is  still  less  than 
g,  then  of  course  the  automaton  will  continue  to  be  in  state  son  and  issue  the  order 
no  action  so  that  the  pump  will  remain  on,  the  plant  trajectory  y{.)  between  tk-^i 
and  tib+2  will  be  strictly  increasing,  and  ^(^*+2)  <  y  +  -I-  e.  We  will  continue  on 

this  way  until  we  find  the  least  I  >  k  such  that  the  measurement  received  at  time  ti 
will  be  greater  than  or  equal  to  g.  By  our  analysis,  the  actual  plant  state  y{ti)  will 
bounded  by  y  +  aA  H-  e.  At  that  point,  the  automaton  will  issue  the  order  for  the 
pump  to  be  turned  off  and  switch  to  state  soff.  What  happens  to  the  trajectory  y{t) 
between  times  ti  and  +  Zi?  It  is  easy  to  see  that  our  analysis  of  Theorem  11 

now  applies.  That  is,  there  will  be  some  ri  <d  <  A  such  that  the  trajectory  satisfies 

0  <  a'  <  y{t)  <a  if  ti  <t  <ti -h  n 
0  >  >  y{t)  >—b  if  -f  71  <  t 


It  is  then  easy  to  see  that  the  trajectory  y{t)  in  the  interval  must  reach 

its  maximum  at  time  t  =  U  ri  and  that  this  maximum  value  is  bounded  by 
y{ti)  +  ari  <  y{ti)  ad  <  g  aA  +  e  +  ad.  Then  after  time  ti  +  77,  y{t)  is  strictly 
decreasing.  It  is  now  easy  to  see  that  if  we  pick  g  so  that 

p  +  od  -f  azl  +  e  <  V, 

then  we  will  ensure  that  following  the  h)  strategy  will  ensure  that  the  water 
level  never  becomes  greater  than  v.  There  is  also  a  lower  bound  which  is  imposed  on 
g  which  comes  from  the  fact  that  the  minimum  value  of  y{t)  in  the  interval 
must  be  greater  than  or  equal  to  u.  Since  we  are  assuming  that  mi  >  g,  we  know 
that  y{ti)  >  ^  —  e.  If  we  assume  that  there  is  no  delay  in  turning  the  pump  off, 
then  y{t)  could  be  strictly  decreasing  in  the  interval.  It  is  then  easy  to  see  that  in 
such  a  situation,  y(t/+i)  could  be  as  small  as  g  —  e  —  bA,  Moreover  it  could  be  that 
g^e  —  bA  —  e<h  so  that  mi^i  <  /i.  In  that  situation,  the  pump  will  be  off  and 
our  controller  would  tell  the  pump  to  turn  on.  However  there  could  be  a  maximum 
delay  of  time  d  before  the  pump  turns  on  and  the  the  water  level  once  again  starts 
to  increase.  Thus  there  could  be  a  further  drop  of  —bd  in  the  water  level  during  this 
delay  so  that  the  water  level  could  become  as  small  as  g  -  e  —  bA  —  bd.  Thus  we 
must  also  assume  that  g  —  bd  —  bA  —  e>u  or  equivalently  that  u  —  bd  +  bA-\-e<g. 
In  case  2,  we  will  deal  with  the  case  when  >  h. 

Case  2.  Suppose  that  the  plants  state  is  soff  and  at  time  t*,  Control  receives 
a  measurement  mk  >  h.  Again  the  actual  water  level  y{tk)  satisfies 

mk-e<  y{tk)  <  4- e. 

Assume  also  that  the  pump  is  off  at  time  tk  so  that  in  this  case  the  automaton 
remains  in  state  soff  and  issues  the  order  no  action  and  the  pump  remains  off  for 
the  next  A  seconds.  Then  since  the  plant  trajectory  y(.)  between  tk  and  tk-\-i  =  tk+A 
must  satisfy 

0  >  -6'  >  y{t)  >  -6, 

it  is  easy  to  see  that  y{t)  is  a  strictly  decreasing  function  in  this  interval  and  that 

y{h+i)  >  y{h)  —  bA  >  mk  —  bA  —  e  >  h  —  bA  —  e. 

Now  if  we  find  that  the  measurement  received  at  time  tk-\.i,  mjfc+i,  is  still  greater 
than  h,  then  of  course  the  automaton  will  continue  to  be  in  state  soff  and  issue  the 
order  no  action  so  that  the  pump  will  remain  off,  the  plant  trajectory  j/(.)  between 
tk-\-i  and  tfc+2  will  be  strictly  decreasing,  and  y{tk^2)  >  h  —  bA  —  e.  We  will  continue 
on  this  way  until  we  find  the  least  I  >  k  such  that  the  measurement  received  at  time 
ti  will  be  less  than  or  equal  to  h.  By  our  analysis,  the  actual  plant  state  y{ti)  will 
bounded  below  hy  h  —  bA  -  e.  At  that  point,  the  automaton  will  issue  the  order 
for  the  pump  to  be  turned  on  and  switch  to  state  son.  Again  use  our  analysis  of 
Theorem  11  to  analyze  what  happens  to  the  trajectory  y{t)  between  times  ti  and 
ti^i  =  ti  A.  That  is,  there  will  be  some  ti  <  d  <  A  such  that  the  trajectory 
satisfies 


0  >  —V  >  y{t)  >  —b  ifti<t<ti’\-ri 
0  <  a*  <  y(t)  <  a  if  t/  4-  r/  <  t 


It  is  then  easy  to  see  that  the  trajectory  y{t)  in  the  interval  must  reach 

its  minimum  at  time  t  —  ti+ri  and  that  this  minimum  value  is  bounded  below  by 
y{ti)  —  bTi>  y{ti)  —  bd  >  h  -  bA  —  e  —  bd.  Then  after  time  +  r;,  y{t)  is  strictly 
increasing.  It  is  now  easy  to  see  that  if  we  pick  h  so  that 

h  —  bd  —  bA  —  e  >  u, 

then  we  will  ensure  that  following  the  A{g,  h)  strategy  will  ensure  that  the  water 
level  never  becomes  less  than  u.  There  is  also  upper  bound  which  is  imposed  on  h 
which  comes  from  the  fact  that  the  maximum  value  of  y{t)  in  the  interval 
must  be  less  than  or  equal  to  v.  Since  we  are  assuming  that  m/  <  h,  we  know  that 
y{U)  <  ft  +  e.  If  we  assume  that  there  is  no  delay  in  turning  the  pump  on,  then 
y{t)  could  be  strictly  increasing  in  the  interval.  It  is  then  easy  to  see  that  in  such  a 
situation,  y{ti^x)  could  be  as  large  as  h-\-e-\-baA,  Note  that  the  case  when  m/+i  <  g 
was  handled  in  Case  1.  However  it  could  be  that  /i*f  e*f  aZ\-f  e  >  ^  so  that  >  g. 
In  that  situation,  the  pump  will  be  on  and  our  controller  would  tell  the  pump  to  turn 
off.  However  there  could  be  a  maximum  delay  of  time  d  before  the  pump  turns  off 
and  the  the  water  level  once  again  starts  to  decrease.  Thus  there  could  be  a  further 
rise  of  ad  in  the  water  level  during  this  delay  so  that  the  water  level  could  become 
as  large  as  /i  4-  e  -f-  +  od.  Thus  we  must  also  assume  that  h-had^aA  +  e  <v  or 

equivalently  that  h  <v  —  ad  —  aA  —  e. 

Below  is  the  proposition  asserting  the  conditions  for  correctness  of  the  A(p,  h) 
control  automaton. 

Theorem  12,  Suppose  in  the  discrete  sampling  game  for  the  water  level  monitor,  we 
have  a  maximum  delay  of  d  for  switching  plant  states,  we  are  given  a  finite  sampling 
time  A>d>0  and  a  measurement  error  bound  e  >  0.  Choose  the  numbers  h  <  g 
so  that 


u  -^-bd-^-bA^  e  <  g,h  <v  —  ad  —  aA  —  e. 

Suppose  that  the  initial  water  level  is  between  h  e  and  v  —  a  *  d  and  the  pump 
is  on  or  the  initial  water  level  is  between  u  +  6  •  d  and  g  —  e  and  the  pump  is  off. 
Suppose  that  initially  the  pump  and  the  control  automaton  are  both  in  the  ^^on”  state 
or  both  in  the  “off”  state.  With  the  A{g,h) -controller  introduced  above,  the  water 
level  satisfies  the  performance  specification  that  u  <  y{t)  <v  at  all  times  t>0. 

Proof  By  using  the  analysis  of  Case  1  and  Case  2  above,  one  can  easily  prove  by 
induction  k  that  if  Control  follows  the  A{g,  h)  strategy  in  our  modified  game,  then 
in  each  interval  [tkytk+i],  the  trajectory  of  the  plant  y{t)  will  always  satisfy  that 
^  <  y{^)  ^  We  leave  the  details  to  the  reader,  □ 

We  note  that  the  inequalities  on  g  and  h  in  Theorem  12  automatically  impose 
the  following  upper  bound  on  the  size  of  the  sampling  interval  A: 

^  ^  V  —  u  +  d(a  H-  6)  -  2e 
fl  -h  6 


4.2  Topological  Finite  Automata  from  Open  Covers 

In  appendix  II  of  [24],  there  is  a  general  method  which,  given  a  hybrid  system  whose 
performance  specification  is  autonomous,  extracts  a  finite  automaton  which  which 
can  be  used  to  guarantee  that  the  hybrid  system  will  meet  its  performance  speci¬ 
fications  as  well  as  to  extract  small  topologies  which  guarantee  the  stability  of  the 
system.  Our  goal  in  this  section  is  to  follow  appendix  II  of  [24]  and  construct  a  finite 
open  cover  yielding  a  finite  control  automaton  and  small  topologies  for  our  water 
level  monitor  example  which  guarantee  that  the  water  level  always  stay  within  speci¬ 
fied  bounds.  Here,  when  we  say  that  the  performance  specification  is  autonomous,  we 
mean  the  following.  We  assume  that  the  plant  is  modeled  by  a  differential  equation 

y-=f{y,u,d) 

where  u  is  a  control  parameter  and  d  is  a  disturbance  parameter.  Then  in  each 
interval  of  time  A  =  [to^ti]  and  any  given  plant  state  y  that  lies  within  a  certain  set 
of  acceptable  values,  we  want  to  find  a  control  law  u(.)  such  that  if  we  use  the  control 
law  u{t)  to  determine  the  plant  trajectory,  then  for  any  acceptable  disturbance  d(t), 
our  plant  trajectory  should  meet  the  required  performance  specification.  That  is, 
any  function  y{t)  such  that  y{to)  =  y  and  y{t)  =  fiy(t),u{t),  d{t))  for  allt  e  A  must 
meet  our  performance  specification.  We  assume  that  our  choice  of  suitable  control 
functions  u{t)  for  any  interval  A  depends  only  on  the  plant  state  x  and  the  internal 
state  of  the  controller  but  not  on  the  time  t  which  is  the  start  of  the  interval. 
In  this  situation,  the  problem  of  meeting  performance  specification  is  equivalent 
to  determining  a  set  Q  of  “acceptable  "pairs  {x,u{t))  of  plants  states  and  control 
functions.  That  is,  each  pair  represents  a  plant  trajectory  which  begins  at  the  plant 
state  of  the  pair  and  is  guided  by  the  control  law  of  the  pair  which  satisfies  the  our 
performance  specifications  over  the  sampling  interval  A.  Note  that  in  this  situation, 
the  control  law  u(t)  is  a  function  of  time  over  the  sampling  interval  that  takes  values 
in  the  range  of  values  of  the  control  parameter. 

For  example,  the  range  of  the  control  parameter  for  the  water  pump-tank  sys¬ 
tem  is  the  set  of  orders  for  the  pump  or  equivalently  the  set  of  states  of  the  pump 
{1  =  ‘pon’,0  =  ‘po//’}  and  every  control  law  is  a  constant  function  over  the  sam¬ 
pling  interval  with  the  range  being  the  pump  states.  In  what  follows,  we  adapt  the 
definition  of  the  set  of  pairs  Q  to  refiect  the  presence  of  possible  delays  in  switching 
the  pump  states.  Thus  for  the  water  tank  and  pump  example,  we  let  Q  consist  of 
the  pairs  such  that  for  any  admissible  delay  in  switching  to  a  new  pump  state  as 
directed  by  Control  in  the  sampling  interval  A,  the  the  water  level  which  correspond 
to  the  plant  trajectory  stays  within  our  required  bounds. 

In  the  general  setting  for  autonomous  performance  specification,  the  first  stage 
of  finding  a  control  automaton  in  the  small  topologies  satisfying  the  specification  is 
to  find  a  control  function. 

Definition  13.  A  feedback  control  function  if  is  a  map  that  assigns  to  each 
pair  of  a  plant  state  x  reached  at  the  end  of  a  sampling  interval  A  and  the  current 
control  law  u  used  in  A,  a  control  law  n'  such  that  the  corresponding  plant  trajectory 
over  the  next  sampling  interval  zi'  satisfies  the  performance  specification  over  that 
interval. 


A  useful  model  to  keep  in  mind  is  to  think  of  the  control  u(t)  as  being  deter¬ 
mined  by  a  physical  controller.  Thus  the  automaton  communicates  with  the  physical 
controller  by  setting  the  state  of  the  physical  controller  Su  which  has  the  effect  of 
imposing  the  control  u(t)  for  the  next  sampling  interval.  In  such  a  situation,  we 
can  identify  the  control  laws  with  the  states  of  the  physical  controller.  For  example, 
in  the  case  of  the  water  pump  and  tank  example  where  the  control  functions  are 
piecewise  constant,  we  may  represent  u  its  value  which  is  either  port  or  poff.  For 
the  rest  of  this  section,  we  shall  use  this  model  so  that  instead  of  talking  about  the 
current  control  law  of  the  sampling  interval,  we  will  talk  about  the  current  state  of 
the  physical  controller,  etc. 

Definition  14.  The  automaton  A{H)  associated  with  a  control  function  H  is  de¬ 
fined  as  follows. 

1.  Its  set  of  states  is  the  set  of  states  of  the  physical  controller  K.  (In  the  more 
general  language,  K  would  be  the  set  of  possible  control  laws  which  occur  in 
pairs  in  Q.) 

2.  Its  input  alphabet  is  the  set  of  plant  states  U  =  PS. 

3.  Its  output  function  H{u^k)  is  the  feedback  control  function. 

4.  Its  transition  table  M{u,  k)  models  the  switching  of  control  laws  output  by  the 
controller,  i.e.  M(u,  k)  =  H{u,  k)  for  all  u  G  C/  and  k  G  K. 

Next  we  want  to  isolate  some  properties  of  the  automaton  A{H)  or  equivalently 
the  feedback  control  function  H  which  will  guarantee  that  we  can  perpetually  apply 
our  control  strategy. 

Definition  15.  Say  that  the  automaton  A{H)  associated  with  a  feedback  control 
function  H  is  correct  with  respect  a  performance  specification  and  a  region 
B  C  PS  X  K  if  the  following  holds.  For  any  pair  a  =  (y^k)  G  B  and  for  any 
admissible  disturbance  d(t),  any  trajectory  beginning  from  y  and  guided  by  the 
control  corresponding  to  k  during  the  delay  for  switching  to  new  state  of  the  physical 
controller  k)  and  by  the  control  k)  after  the  delay  satisfies  the  performance 
specifications  and  ends  up  in  B  at  the  end  of  the  sampling  interval.  Here  “ends  up 
in  J5”  means  that  if  yi  is  plant  state  corresponding  the  trajectory  at  the  end  of  the 
sampling  interval,  then  {yi,H{y,k))  G  B. 

Definition  16.  Suppose  that  there  is  a  region  B  in  the  domain  of  the  feedback  con¬ 
trol  function  H  such  that  for  any  pair  (y,  k)  G  B  and  for  any  admissible  disturbance 
d(t),  any  trajectory  beginning  from  y  and  guided  by  the  control  corresponding  to  k 
during  the  delay  for  switching  to  new  state  of  the  physical  controller  H{y^k)  and 
by  the  control  control  H{y,k)  after  the  delay  satisfies  the  performance  specifications 
and  ends  up  in  B  at  the  end  of  the  sampling  interval.  Then  we  call  such  a  control 
function  a  guiding  feedback  control  function  relative  to  B. 

The  definitions  above  can  easily  be  extended  to  apply  to  the  case  when  the  control 
function  is  set- valued  as  introduced  in  appendix  II  of  [24].  The  idea  of  a  set- valued 
feedback  control  function  is  that  one  computes  a  set  of  controls  or  in  our  case  a  set 
of  physical  controller  states  from  a  pair  consisting  of  a  plant  state  and  a  physical 


controller  state  and  then  selects  from  that  set  one  control  or  physical  controller  state 
which  will  be  used  to  determine  the  plant  trajectory  in  the  next  sampling  interval 
A.  The  set  of  control  functions  or  physical  controller  states  that  we  compute  should 
be  such  that  for  every  control  function  or  physical  controller  state  that  could  have 
been  chosen  from  the  set  and  every  admissible  disturbance,  the  corresponding  plant 
trajectory  always  satisfies  the  performance  specifications. 

Set- valued  feedback  control  functions  arise  naturally  in  our  context.  Consider  a 
map  H  from  the  pairs  (m,  fc)  ((measurement,  physical  controller  state))  into  the  set  of 
states  of  the  physical  controller.  If  we  take  the  measurements  as  inputs  to  the  control 
automaton  and  identify  the  map  If  with  the  control  automaton  output  function, 
we  of  course  have  an  ordinary  function  as  opposed  to  a  set-valued  one.  However 
suppose  that  we  assume  that  a  measurement  can  be  any  value  that  approximates  a 
plant  state  within  some  error  bound.  That  is,  we  view  a  measurement  as  a  set-valued 
function  over  plant  states  from  which  an  nondeterministic  choice  of  an  element  from 
a  set  is  made.  For  example,  suppose  that  the  map  above  is  II(m,k),  where  m  is  a 
measurement,  and  kisa  physical  controller  state.  Then  the  corresponding  set-valued 
feedback  control  function  is  G(y,k)  =  {If(m,k)  :  {m  —  y\  <  e}.  Here  e  >  0  is  the 
measurement  error  bound. 

In  appendix  II  [24],  the  graph  of  G  is  assumed  closed.  But  our  G  is  not  closed. 
So  we  take  the  closure  of  the  graph  of  G  and  consider  a  corresponding  set-valued 
function  G'  whose  graph  is  that  closure.  So  our  control  function  will  be  G'.  The 
topologies  that  are  used  in  the  construction  of  G'  are  the  natural  Hausdorff  topologies 
on  the  plant  state  space  and  on  the  space  of  states  of  the  physical  controller  following 
[24].  The  fact  that  the  topological  spaces  are  Hausdorff  means  that  if  the  state  space 
K  of  the  physical  controller  is  finite,  then  the  K  must  have  the  discrete  topology 
since  the  only  Huassdorff  topology  on  a  finite  set  is  the  discrete  topology. 

It  is  also  important  for  applying  the  methodology  of  appendix  II  that  the  domain 
of  the  feedback  control  function  be  a  subset  of  the  set  Q.  This  is  true  of  the  graph 
of  G  but  not  necessarily  for  the  closure  of  G  because  the  domain  of  the  closure  of  G 
may  include  boundary  points  of  Q  which  are  not  in  Q,  In  the  case  we  consider,  the 
closure  of  G  will  in  fact  lie  entirely  in  Q. 

Now  let  us  go  back  to  our  water  level  monitor  example.  Let  K  =  {pon,poff}  be 
the  range  of  control  values  or  equivalently  the  states  of  the  pump.  Let  the  variable 
k  range  over  the  set  K,  Here,  the  map  (m,  k)  is  defined  by 

“Lite 

A  water  level  y  is  taken  from  the  set  [u,t;],  which  carries  the  natural  Euclidean 
topology.  There  is  only  one  Hausdorff  topology  on  the  set  K,  the  discrete  topology. 

To  construct  the  function  G  note  that  for  each  control  automaton  state  the  func¬ 
tion  H  is  continuous  except  at  one  point  in  the  range  of  y.  The  point  of  discontinuity 
for  H  is  either  g  or  h  at  respective  automaton  states  on,  off.  It  follows  that  if  y 
is  separated  from,  say  g,  by  more  than  the  error  bound  e,  then  the  function  has  a 
singleton  set  as  a  value.  One  can  see  that  at  points  ^  —  e,  ^  -h  e,  ft  —  e,  h  -h  e,  the  value 


of  G  is  still  a  singleton.  At  points  near  to  g,  h  by  less  than  e,  the  value  of  G  is  K 
since  H  has  a  different  value  to  the  right  of  g  than  to  the  left  of  g.  Thus 

{{port}  ify<g-e 
K  if  g-e<y<g-{-e 
{poff}  if  y>g  +  e 

{{port}  ify<h  —  e 
K  if/i-e<T/</iH-e 
{poff}  if  2/  >  h  +  e 

Now  consider  the  closure  G'  of  the  graph  of  G.  Here  we  use  the  same  letter  for  the 
set-valued  function  and  for  its  graph.  Here  is  the  resulting  closure. 

{{pon}  ify<g-e 
K  if  g-e<y  <g  +  e 
{poff}  if  y>g^e 

{{pan}  ify<h  —  e 
K  if  h-e<y  <h-\-e 
{poff}  if  2/  >  h  -h  e 

Note  that  the  definition  of  the  function  G  also  makes  sense  for  exact  measurements 
(e  =  0),  but  in  that  case  the  corresponding  function  G'  is  multi-valued  only  at  the 
points  of  discontinuity  (^,on),  (h,o//)  of  H,  This  nondeterminacy  makes  clear  the 
arbitrary  nature  of  the  choice  of  a  strict  or  non  strict  inequality  in  the  definition  of 
H,  That  is,  we  obtain  four  functions  which  are  variants  of  if,  differing  from  if  only 
in  having  non-strict  inequalities  in  the  definition.  All  give  rise  to  the  same  G'. 

We  distinguish  between  three  slightly  different  automata,  Auti^  Aut2^  and  Auts, 
which  depend  on  our  pair  of  parameters  g  and  /i.  For  all  three  automaton,  the 
set  of  states  is  {pon,po//},  the  input  alphabet  is  the  set  of  water  levels  and  the 
output  alphabet  the  same  as  the  set  of  states.  Thus  we  need  only  define  their  output 
functions  ift(2/j  fc)  and  their  transition  tables  Mi{y^  k).  For  the  automaton  Auti{g^  h), 
the  output  function  Hi{yy  k)  and  the  transition  table  Mi (2/,  k)  are  both  equal  to  the 
function  H{yyk)  defined  above.  If  we  think  of  this  automaton  as  a  strategy  for 
Control  in  the  discrete  sampling  game  with  error  measurements,  then  Auti  gives 
essentially  the  same  strategy  as  the  automaton  A(^,  h)  described  in  the  previous 
section.  The  only  difference  between  the  two  automaton  is  when  in  the  state  pon 
when  y  <  gy  Autx{gyh)  outputs  pon  while  A(p,/i)  outputs  no  action.  However  we 
regard  both  of  these  instructions  to  a  pump  which  is  on  to  be  the  same,  i.e.  they  both 
keep  the  pump  on.  Similarly  when  in  the  state  poff  when  y  >hy  Auti{g,  h)  outputs 
poff  while  A{gy  h)  outputs  no  action.  Again  we  regard  both  of  these  instructions 
to  a  pump  which  is  off  to  be  the  same,  i.e.  they  both  keep  the  pump  off.  Thus 
by  Proposition  12  Auti  is  a  winning  strategy  for  Control  in  the  discrete  sampling 
game  with  error  measurements.  Now  as  observed  above,  if  we  think  about  the  action 
of  the  strategy  as  a  function  of  plant  states  instead  of  on  measurements  where 
we  assume  that  the  absolute  value  of  the  difference  between  the  measurement  and 
the  actual  plant  state  is  no  more  than  e,  then  the  transition  table  and  the  output 


function  axe  nondeterministic  and  are  give  by  the  function  G  defined  above.  Thus 
we  define  a  second  automaton  Aut^ig^  h)  whose  transition  table  and  output  function 
are  given  by  G,  i.e.  for  all  {y,  fc),  M2(y,  k)  =  H2{y,  k)  =  G{yj  k).  Of  course  Aut2  is  a 
nondeterministic  automaton  and  the  output  function  is  set  valued.  We  shall  assume 
that  the  automaton  operates  as  follows.  If  Aut2  is  in  state  s  and  is  reading  input  y 
and  goes  to  state  s'  at  its  next  step  so  that  s'  €  M2(s,y),  then  the  output  of  the 
automaton  in  that  circumstance  is  also  s'.  That  is,  our  definitions  ensure  that  for  the 
pair  (s,2/),  the  possible  new  states  and  the  possible  outputs  come  from  the  same  set 
since  M2(s,2/)  =  H2{s,y).  We  are  thus  making  the  additional  assumption  that  such 
choices  are  coordinated  for  any  (s,  y).  In  this  way,  we  can  use  Aut2  as  a  strategy  for 
Control  since  our  assumption  will  ensure  that  the  internal  state  of  the  automaton 
Auti  and  the  state  of  pump  are  always  coordinated  at  the  end  of  sampling  intervals 
if  they  start  out  coordinated.  If  we  think  of  Aut2  as  a  strategy  for  Control  in  the 
discrete  sampling  game  without  errors  in  measurements,  i.e.  in  the  discrete  sampling 
game  where  the  error  bound  e  =  0,  then  this  strategy  for  Control  will  produce  exactly 
the  same  set  of  runs  with  respect  to  plant  states  as  the  strategy  Auti  produces  in  the 
discrete  sampling  game  with  error  measurements.  Hence  Aut2  is  a  winning  strategy 
for  control  in  the  discrete  sampling  games  without  error  measurements.  Finally  we 
consider  yet  another  nondeterministic  automaton  Auts  whose  transition  table  and 
output  function  are  given  by  G'  instead  of  G.  Again  we  assume  that  Aut^  operates 
so  that  if  Autz  is  in  state  s  and  is  reading  input  y  and  goes  to  state  s'  G  Mz{s, y)  at 
its  next  step,  then  the  output  of  the  automaton  in  that  circumstance  is  also  s'. 
Remark 

The  differences  between  the  control  strategy  Auti  in  our  discrete  sampling  game 
with  errors  in  measurements  bounded  by  e  and  the  control  strategy  Aut2  in  our  dis¬ 
crete  sampling  game  without  error  measurements  can  be  explained  in  terms  whether 
we  consider  the  analog  to  digital  converter  as  part  of  the  plant  or  whether  we  want  to 
consider  the  analog  to  digital  converter  as  part  of  the  digital  controller.  That  is,  if  we 
consider  the  analog  to  digital  converter  as  part  of  the  plant,  then  it  is  natural  to  as¬ 
sume  that  the  digital  controller  receives  only  plant  measurements  and  this  situation 
is  most  naturally  modeled  as  a  discrete  sampling  game  with  errors  in  measurements 
where  the  control  automaton  is  deterministic.  However,  if  we  consider  the  analog 
to  digital  converter  as  part  of  the  digital  controller,  then  the  most  natural  way  to 
model  this  situation  is  that  we  have  a  discrete  sampling  game  without  errors  in  mea¬ 
surements  and  that  the  control  automaton  behaves  in  a  nondeterministic  manner  as 
described  by  Aut2*  Thus  our  choice  of  using  Auti  in  a  discrete  sampling  game  with 
errors  in  measurements  or  of  using  Aut2  in  a  discrete  sampling  game  without  errors 
in  measurements  comes  down  to  the  choice  of  where  in  Figure  1  we  wish  to  place 
the  analog  to  digital  converter,  i.e  on  the  digital  side  or  on  the  analog  side. 

Our  next  proposition  states  that  Autz  is  also  a  winning  strategy  for  Control  in 
the  discrete  sampling  games  without  error  measurements. 

Theorem  17.  Suppose  in  the  discrete  sampling  game  without  errors  in  measure¬ 
ments  for  the  water  level  monitor,  we  use  finite  sampling  intervals  of  size  A  and 
that  the  maximum  delay  d  for  switching  to  new  plant  state  is  such  that  A  >  d>  0. 
In  addition  assume  e  >  0  and  that  g  and  h  satisfy 

1,  gKV’-a^d  — a-A  — e; 


2,  g  —  e  >  h  e; 

3.  h^u-\-b'd-\-b'A-\-e. 

Suppose  that  the  initial  water  level  is  between  /iH-e  and  v  —  o  •  d  and  the  pump  is 
on,  or  the  initial  water  level  is  between  u+b*d  and  g—e  and  the  pump  is  off.  Suppose 
that  initially  the  pump  and  any  of  the  two  control  automata,  Aut2  or  Aut3,  are  both 
in  the  on  state  or  both  in  the  off  state.  Then  Aut2  and  Auts  are  winning  strategies 
for  Control  in  such  discrete  sampling  games  without  errors  in  measurements  for  the 
water  level  monitor. 

Proof  The  proof  of  Theorem  12  that  Autl  is  a  winning  strategy  for  Control  in 
discrete  sampling  games  with  errors  in  measurement  bounded  by  e  can  be  is  easily 
adapted  to  prove  that  that  either  Aut2  or  Aut3  is  a  winning  strategy  for  Control 
in  the  discrete  sampling  games  without  errors  in  measurement.  The  proof  is  by 
induction  on  the  length  of  positions  as  before.  We  leave  the  details  to  the  reader.  □ 

The  content  of  Theorem  17  can  be  restated  as  the  following  property  of  the 
feedback  control  function  G'.  Suppose  the  water  level  y  is  between  h  +  e  and  v  — a- d 
and  the  pump  is  on  or  y  is  between  u  -f  6  •  d  and  g  —  e  and  the  pump  is  off.  Suppose 
that  the  next  control  law  is  chosen  from  the  set  G^{y,k),  where  k  is  the  state  of 
the  pump  as  specified  above  at  the  beginning  of  the  sampling  interval  A.  Then  the 
water  level  lies  in  the  interval  [u,v]  over  the  next  sampling  interval.  Thus  G'  can 
indeed  be  used  as  a  feedback  control  function  for  the  water  level  and  pump  states 
in  the  region 


A  ^  [h  e,v  —  a  *  d]  X  {on}  [J[u  +  b'd,g  —  e]  x  {off}. 

Moreover,  the  water  level  and  the  state  of  the  pump  at  the  end  of  the  sampling 
interval  satisfy  the  same  assumptions  that  are  satisfied  by  this  data  at  the  beginning 
of  the  sampling  interval.  That  is,  the  trajectories  that  have  begun  in  A  will  end  in 
A  at  the  end  of  a  sampling  interval  if  they  are  guided  by  a  control  law  determined 
by  the  set-valued  control  function.  According  to  our  earlier  definition,  the  feedback 
control  function  G'  restricted  to  A  is  a  guiding  feedback  control  function. 
Constructing  Open  Covers 

We  now  consider  an  open  cover  of  the  graph  of  G'  restricted  to  the  region  A. 
Our  goal  is  to  construct  a  finite  automaton  with  small  topologies  approximating 
G'.  We  presented  A  above  as  a  disjoint  union  of  two  open  and  closed  (clopen)  sets. 
Correspondingly,  the  graph  of  G'  is  a  disjoint  union  of  clopen  sets.  It  is  sufficient 
to  cover  each  of  the  clopen  sets  independently.  Choose  e  >  0  so  small  that  the  sets 
below  are  subsets  of  the  graph  of  G'.  To  visualize  the  regions  below  more  clearly, 
recall  that  we  have  the  following  the  inequalities: 

h  —  e<g  —  e<g  +  e<v  —  C'd  and 
u-\-b'd<h  —  e<h-\-e<g  —  e. 

Here  is  the  open  cover  for  the  first  clopen  set: 

Vi  =  [h  +  e,  ^  -  e  -h  c)  X  {on}  x  {on}, 

V2  =  (^  -  e  -  e,^  +  e  +  e)  X  {on}  x  K, 

V3  {g  e  -  €,v  -  a  •  d\  X  {on}  x  {off}. 


Similarly  here  is  an  open  cover  for  the  second  clopen  set: 

14  =  [«  +  6'd,/i-e  +  €)x  {off}  X  {on}, 

^5  =  (h  -  e  -  +  e  +  e)  X  {off}  x  K, 

V6  =  {h  +  e-€,g  +  e]x  {off}  x  {off}. 

Let  Ui,U2,U3,U4,U5,Uq  be  the  leftmost  components  of  VsjVe  re¬ 

spectively.  The  input  alphabet  of  the  small  topologies  automaton  will  consist  of  the 
two  disjoint  lists.  Namely  the  join  irreducibles  of  the  lattice  under  inclusion  gener¬ 
ated  by  Uiy  U2  and  U3  which  consist  of 

UiyU2,U3yUinU2,U2nU3 

and  the  set  of  join  irreducibles  of  the  lattice  under  inclusion  generated  by  1/4,  U5  and 
Ue  which  consists  of 

U4,Us,UeyUAnUs,UsnUQ, 

In  the  notation  of  [24],  the  sets  Vi,  i  =  1, . . . ,  6,  correspond  to  an  open  cover  of 
the  graph  of  G'  restricted  to  A.  The  sets  of  the  cover  are  of  the  form  Vi  =  Ai  x  Biy 
with  1  <  i  <  6  where 

Ai  =  Ui  X  {on}  1  <  i  <  3, 

Bi  =  {on},  B2  ^KyB3  =  and 

Ai  =  UiX  {off},  3  <  i  <  6, 

B4  =  {on},  Bs  =  K,  Be  =  {off}. 

The  finite  automaton  in  the  small  topologies  described  in  [24]  assigns  to  each 
join-irreducible  in  the  lattice  generated  by  the  open  sets  A*,  a  set  of  control  laws. 
That  is,  we  attach  to  every  non-empty  join  irreducible  in  the  lattice  generated 
by  the  Ai’s,  an  open  set 

0{A[)  —  UzeFi^zy 

where  Ji  =  {2  |  A\  C  Az}- 

In  our  case  it  is  easy  to  check  that  we  obtain  the  following  assignments  of  0{A[) 
for  the  join-irreducibles  A[: 

1.  the  sets  Ui  x  {on},  1  <  i  <  3  are  mapped  respectively  to  {on}, If,  {off}, 

2.  the  sets  Ui  x  {off},  3  <  i  <  6  are  mapped  respectively  to  {on},  if,  {off}, 

3.  each  of  the  following  four  join  irreducibles,  {Ui  D  U2)  x  {on}),  {U2  H  t/3)  x  on}, 
(C/4  n  C/5)  X  {0//},  (C/5  n  C/e)  X  {off},  is  mapped  to  if. 

Let  H{u,  k)  be  any  set-valued  function  which  is  consistent  with  the  above  assign¬ 
ments  where  u  ranges  over  the  set  U  of  join  irreducibles  in  the  lattices  generated  by 
C^i>  C/^2j  U3  and  by  C/4,  C/5,  C/e .  Formally,  the  finite  automaton  in  the  small  topologies 
corresponding  to  the  above  data  is  the  following: 

1.  The  set  of  states  5  =  if . 

2.  The  input  alphabet  is  the  set  U, 

3.  The  output  alphabet  V  —  K, 

4.  The  nondeterministic  output  function  is  based  on  the  set-valued  function  H 
described  in  the  assignments  above. 

5.  The  transition  table  M  :  C/  x  if  if  is  defined  by  M{u,  k)  =  H{u,  k). 


The  automaton  can  be  used  for  control  as  follows.  Let  y  be  a  water  level  and  k 
be  the  automaton  current  state. 

1.  The  analog  to  digital  converter  transforms  y  into  the  least  join-irreducible  u 
that  contains  y. 

2.  The  automaton  maps  u  nondeterministically  into  a  pump  state  A:'  G  H{u,k), 
and  outputs  fc'  to  the  plant. 

This  automaton  is  parameterized  by  the  e  entering  the  definitions  of  t/^^  Are  there 
values  of  e  which  guarantee  that  water  level  trajectories  arising  firom  the  automaton 
satisfy  the  control  requirements?  While  considering  this  question  we  may  ask  whether 
the  automaton  output  function  is  related  to  a  suitable  feedback  control  function. 
Should  it  happen  to  be  a  guiding  feedback  control  function  for  some  region  of  Q, 
then  the  control  automaton  would  satisfy  the  control  requirements  if  it  began  its 
operation  in  that  region. 

Consider  the  following  “feedback  control  function”:  f{y,k)  =  H{u,k),  where  u 
is  the  least  join-irreducible  that  contains  y.  It  is  then  easy  to  see  that: 

When  k  =  on: 

{K  if  y  G  (y-e-e,y  +  e-fe) 
on  if  y  G  [h  -  e,  y  -  e  -  e] 
off  ifyG[y-he>fc,u-a-cf] 

When  k  =  off, 

{K  if  y  G  (h  e  —  /i  4*  e  “h  e) 
on  if  y  G  [u  4  6  •  d,  h  -  e  -  e] 
off  if  y  G  [/i4e4e,y4e] 

We  have  three  objects  now:  the  finite  automaton  in  the  small  topologies,  the 
corresponding  function  /,  and  the  control  automaton  associated  with  /.  It  is  easy 
to  see  that  each  of  the  three  objects  have  the  same  set  of  water  level  trajectories 
over  the  region  A  generated  by  the  object.  It  follows  that  if  /  is  a  guiding  feedback 
control  function  over  A,  then  the  finite  automaton  with  small  topologies  is  correct. 

We  can  conclude  that  /  is  a  guiding  feedback  control  function  from  the  following 
general  fact  and  Theorem  17. 

Proposition  18,  Suppose  A  C  PS  x  K  and  f,F  are  two  set-valued  functions  over 
A  with  values  subsets  of  K.  Suppose  that  the  graph  of  f  is  a  subset  of  the  graph  of 
F  and  F  is  a  guiding  feedback  control  function.  Then  so  is  f. 

Proof  It  is  clear  that  all  the  plant  trajectories  generated  by  /  constitute  a 
subset  of  those  generated  by  F.  The  conclusion  desired  is  immediate.  □ 

Consider  F,  which  is  determined  by  g,  h  and  e'  =  e4e.  Assume  that  the  premises 
of  Theorem  17  are  satisfied  by  this  data  for  some  eo  >  0.  It  follows  from  Theorem 
17  that  F  is  a  guiding  control  function.  It  follows  from  the  proposition  above  that 
so  is  /  for  any  e  <  cq- 

Remark  The  control  automaton  Autl  described  above  is  a  formal  represen¬ 
tation  of  the  controller  from  [1].  That  paper  does  not  mention  using  a  sampling 


interval  A  >  0,  We  can  interpret  this  as  meaning  that  water  level  is  measured  and 
tested  continuously.  Continuous  measurement  and  testing  in  the  presence  of  pump 
delay  can  cause  the  above  control  automaton  and  the  controller  from  [1]  to  produce 
an  infinite  number  of  outputs  in  a  finite  interval  of  time,  a  physical  impossibility. 
Consider  a  time  t  at  which  the  automaton  outputs  a  request  to  change  the  pump 
state.  Suppose  that  just  prior  to  that  time  the  pump  was  “on”  and  the  state  of  the 
automaton  was  son.  Suppose  that  the  pump  delay  is  d  >  0.  Since  the  water  level 
continues  to  increase  during  the  delay,  and  the  automaton  continuously  samples  the 
input,  the  automaton  senses  the  condition  y  >  g  at  all  times  in  the  interval  (t,  t -fd). 
Thus  the  automaton  will  produce  an  essential  output  at  each  time  in  that  inter¬ 
val.  Our  assumption  that  we  sample  (measure,  sense)  after  each  interval  of  length 
A  >  d  >  0  eliminates  this  source  of  unrealizable  behavior.  Sampling  at  times  sep¬ 
arated  by  a  positive  bound  Zi  >  0  cannot  be  dispensed  in  modeling  a  plant  with 
delays. 

Later  papers  will  investigate  open  covers  and  the  corresponding  finite  automata 
with  small  topologies  for  a  variety  of  control  problems. 
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